Multiomic random forest toxicity modeling of radiation esophagitis

Graphical abstract

Highlights

• •First study to integrate clinical, PET/CT radiomic, dosimetric, and dosiomic features to predict grade ≥3 RE.
• •First to model RE using radiomics from both CT and PET imaging.
• •Multiomic models outperform the DVH model in predicting high-grade RE.

Abstract

Background and purpose

Radiation esophagitis (RE) is a major dose-limiting toxicity resulting from radiotherapy for non-small-cell lung cancer (NSCLC). Multiomic features may provide additional predictive value for high-grade RE compared to traditionally used clinical and dose-volume histogram (DVH) parameters. We aimed to investigate the utility of multiomic features in improving RE toxicity prediction models.

Materials and methods

Of the 179 NSCLC patients considered, 27 patients (15.08 %) were found to have toxicity ≥grade 3 RE per CTCAE v5.0. A total of 343 CT- and PET- based radiomic and dosiomic features were extracted. Four toxicity prediction models were created using clinical factors and features from one of the following groups: (a) base model (DVH), (b) radiomic, (c) dosiomic, and (d) combined radiomic and dosiomic. Models were developed using a random forest classifier with 100 Monte Carlo cross-validation iterations and an 80 %/20 % training/test split. Model predictive performance was evaluated by area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPRC).

Results

The AUC and AUPRC values (mean ± standard deviation) for the 4 model types were 0.69 ± 0.10/0.42 ± 0.12 (base model), 0.71 ± 0.12/0.48 ± 0.13 (radiomic), 0.73 ± 0.10/0.48 ± 0.15 (dosiomic), and 0.75 ± 0.10/0.49 ± 0.14 (radiomic and dosiomic), respectively. The bootstrap percentile method, which was used to compare performance metrics between multiomic and base models, showed that the combined model was the best performing model type.

Conclusion

All multiomic models outperformed the base model. The combined radiomic-dosiomic model provides novel insights into high-grade RE risk and may inform future strategies for toxicity mitigation and personalized treatment planning.

1. Introduction

Lung cancer is the leading cause of cancer-related mortality worldwide [1]. Non-small-cell lung cancer (NSCLC) accounts for 85 % of all lung cancer cases [2]. Radiation therapy (RT) plays a crucial role in the management of unresectable and locally advanced NSCLC, as RT is the primary treatment modality for inoperable NSCLC. Even though higher radiation doses lead to better tumor control [3], dose escalation [4,5] is greatly limited by radiation-induced toxicities such as radiation esophagitis (RE). Typically, patients undergoing radiotherapy for NSCLC develop grade 2 or higher acute RE [6]. This is one of the major toxicities associated with NSCLC and generally occurs within two months after the first radiotherapy session, degrading patient quality of life [7]. Severe esophagitis has the potential to cause treatment interruptions, hospitalization, or parenteral feeding [8]. Ineffective therapeutics and persistent RE remain among the top dose-limiting toxicities that influence RT planning [9]. Various clinical factors [10,11] as well as features derived from dose-volume histograms (DVHs) [[12], [13], [14]] have been used to predict the incidence of RE. However, clinical and DVH feature-based toxicity models have been shown to have suboptimal predictive performance [15,16].

Machine learning has recently emerged in radiation oncology for modeling radiation-induced toxicity outcomes [[17], [18], [19], [20]]. A random forest machine learning classification approach has been shown to outperform other algorithms for outcome predictions with clinical data among some of the widely used machine learning classification approaches [21]. A process termed radiomics involves extracting quantitative features from medical images such as CT and PET scans. With the aid of machine learning, these features can be used to predict therapeutic responses [[22], [23], [24]]. Recently, dosiomics, which is similar to radiomic analyses but instead uses 3D dose distributions as the feature space, has been introduced into outcome prediction modeling (OPM) analyses. These omic features are derived from the extracted radiotherapy dose distributions of patients to predict therapeutic responses. It is possible that dosiomic variables can better predict RE than DVH metrics since they retain the 3D information of the radiation dose distribution as shown by Ma et al. [25] Although some studies have used clinical and radiomic features from CT images to predict RE ≥ grade 2 [11,26], no studies have focused on predicting the more severe RE ≥ grade 3 using multiomic features. While grade 2 esophagitis is considered symptomatic, a higher-grade esophagitis leads to tube feeding and hospitalization [27], thereby severely impacting the quality of life. Therefore, a model that can efficiently predict higher-grade RE would be more clinically relevant.

This study aims to bridge this gap by combining clinical, dosimetric, dosiomic, and radiomic features from both CT and PET scans to predict RE ≥ grade 3. By leveraging shape, statistical, and textural patterns from CT, incorporating 3D dose distributions, and including additional information from PET imaging, we aimed to create a more comprehensive dataset to enhance outcome prediction models.

2. Materials and methods

2.1. Patient data

This retrospective single-center study was approved by the institutional review board. We included 179 NSCLC patients treated at our institution between October 2006 and July 2016 with either intensity-modulated radiation therapy/volume-modulated arc therapy (IMRT/VMAT, n = 91), passive-scatter proton therapy (PSPT, n = 55), or intensity-modulated proton therapy (IMPT, n = 33). All patients received fractionated doses between 1.8–2.6 Gy per fraction for total doses between 60 and 74 Gy. The study exclusion criteria were: (a) presence of acute lung infection, (b) prior thoracic surgery, (c) patients with <6-month follow-up period unless toxicity was noted, (d) patients with a previous history of thoracic radiation therapy. All patients were treated with either induction, concurrent, or adjuvant chemotherapy in addition to RT.

2.2. Toxicity evaluation

Radiation esophagitis was graded from 0 to 5 according to the Common Terminology Criteria for Adverse Events (CTCAE v5.0) [27]. For this study, grade ≥3 esophagitis was used as the binary endpoint.

2.3. Data analysis workflow

The data analysis workflow is illustrated in Fig. 1. A total of 343 features (14 clinical, 14 dosimetric, 105 dosiomic, and 210 radiomic) were extracted, with the esophagus minus the GTV as the ROI. Radiomic features were derived from CT and PET, while dosiomic features were derived from 3D dose distributions. PET scans were converted to standardized uptake values (SUVs) normalized by body weight prior to feature extraction. Feature extraction followed IBSI guidelines [28] using Pyradiomics (v3.0.1) [29], RayStation, and 3D Slicer [30]. After all features were extracted, redundant features were filtered using a supervised Spearman correlation approach applied to the training set [31], and predictive toxicity models were developed with a random forest classifier across four categories: (a) base DVH, (b) radiomic, (c) dosiomic, and (d) combined radiomic–dosiomic. Model performance was then compared. Full details of imaging parameters, SUV conversion for PET scans, radiomic settings, and feature definitions are provided in the Supplemental materials.

Fig. 1.

Overall Study Workflow. DVH, radiomic and dosiomic features are extracted, followed by feature reductions in the first step. Four OPM types are built and performance metrics between the DVH and omic model types are compared in the second step. Abbreviations: PET, positron emission tomography; CT, computed tomography; DVH, dose-volume histograms; ROI, region of interest; AUC, area under the curve; AUPRC, area under precision-recall curve; Prec, precision; Rec, recall; F-1 Score, harmonic mean of precision and recall; CV, cross-validation.

2.4. Model building

The process of model construction and performance metric evaluation is illustrated in Fig. 1. For this study, we created four types of toxicity prediction models using clinical factors, along with one of the following feature sets: (a) base (DVH), (b) radiomic, (c) dosiomic, and (d) combined radiomic and dosiomic model. The random forest (RF) classification model has previously demonstrated strong discriminative performance in radiotherapy outcomes and toxicity predictions and was therefore used in this study [21]. The hyperparameters for the random forest modeling: mtry (controls how much randomness is added to the decision tree creation process), maxnodes (maximum number of terminal nodes allowed in a tree), and ntrees (number of decision trees combined to create the final prediction), were adjusted based on the number of covariates in each of the four models [32].

Toxicity models were developed using a repeated cross-validation approach with 100 iterations. This type of cross-validation is a common method used to approximate the generalizability of the modeling process [19,33,34]. Each iteration of cross-validation generated a unique model, resulting in 100 distinct models for each model type. The same 100 data splits were used to develop all four predictive models to enable paired comparisons. This strategy allowed the statistical significance of performance metrics to be assessed using the distribution of results across model types. For each iteration, the cohort was randomly divided into a training set (80 %) and a test set (20 %). Correlation-based feature reduction was performed on the training set to identify the optimal features. The remaining features were then used for RF modeling for the specific model type.

2.5. Model evaluation

After selecting the optimal hyperparameters using the training set, the model's performance was evaluated on the test set. The following metrics were used to assess each model’s performance across cross-validation test folds: accuracy, area under the receiver operating characteristic curve (AUC), area under the precision-recall curve (AUPRC), precision, recall, and F1-score. Standard deviations were also computed for performance metric distributions across the repeated cross-validation on both the training and test sets. Statistical significance for differences in each performance metric between omic models and the base model was assessed using the bootstrap percentile method. A difference was considered statistically significant if the 95 % confidence interval for the mean difference between models did not include zero. Specifically, after calculating the mean difference in each performance metric between the omic model and the base model across all cross-validation iterations, we used bootstrap resampling (10,000 replicates) to generate an empirical distribution of the mean difference. The 95 % confidence interval was then calculated from this bootstrap distribution by taking the 2.5th and 97.5th percentiles. If the entire confidence interval was greater than zero, the observed difference was statistically significant. To account for multiple comparisons when evaluating differences in model performance, Bonferroni-adjusted bootstrap percentile confidence intervals (98.33 % CI for three comparisons) were applied to the results in Table 1. All statistical analyses were performed using RStudio (v4.2.1) (Posit, PBC; Boston, Massachusetts). The open-source R package “caret,” along with randomForest package, was used to build all machine learning outcome prediction models. To test the model’s robustness, we introduced substantial Gaussian noise to all CT, PET, and dose images and extracted multiomic features from the perturbed images. After filtering for reliable multiomic features (with an ICC > 0.75) and comparing model performance using all multiomic features versus only robust features, we found that the model performance metrics remained consistent. This supports the robustness of our modeling approach. Detailed information about the robustness analysis is provided in the Supplemental materials.

Table 1.

Results of test set evaluation for different model types, reported as the mean of each metric with one standard deviation. Comparisons of radiomic, dosiomic, and combined radiomic–dosiomic models with the base model (DVH metrics) were conducted using the bootstrap percentile method. Confidence intervals for the mean difference were estimated from 10,000 bootstrap resamples. The dosiomic and combined model types demonstrated improved AUC and AUPRC compared to the base model, with the combined radiomic–dosiomic model outperforming all other model types.

A. Radiomic vs Base
Performance Metrics	Base	Radiomic	95 % CI
Accuracy	0.87 ± 0.02	0.88 ± 0.02*+	[0.001,0.012]
AUC	0.69 ± 0.10	0.71 ± 0.12	[-0.01,0.05]
AUPRC	0.42 ± 0.12	0.48 ± 0.13*+	[0.03,0.10]
Precision	0.39 ± 0.18	0.46 ± 0.19*+	[0.01,0.12]
Recall	0.69 ± 0.24	0.69 ± 0.20	[-0.06,0.06]
F1-Score	0.45 ± 0.10	0.50 ± 0.11*+	[0.03,0.09]
B. Dosiomic vs Base
Performance Metrics	Base	Dosiomic	95 % CI
Accuracy	0.87 ± 0.02	0.88 ± 0.02*+	[0.002,0.01]
AUC	0.69 ± 0.10	0.73 ± 0.10*+	[0.02,0.07]
AUPRC	0.42 ± 0.12	0.48 ± 0.15*+	[0.02,0.10]
Precision	0.39 ± 0.18	0.46 ± 0.21*+	[0.02,0.12]
Recall	0.69 ± 0.24	0.69 ± 0.21	[-0.05,0.06]
F1-Score	0.45 ± 0.10	0.50 ± 0.12*+	[0.02,0.09]
C. Radiomic + Dosiomic vs Base
Performance Metrics	Base	Radiomic & Dosiomic	95 % CI
Accuracy	0.87 ± 0.02	0.88 ± 0.02*+	[0.008,0.01]
AUC	0.69 ± 0.10	0.75 ± 0.10*+	[0.03,0.09]
AUPRC	0.42 ± 0.12	0.49 + 0.14*+	[0.03,0.10]
Precision	0.39 ± 0.18	0.48 + 0.20*+	[0.04,0.09]
Recall	0.69 ± 0.24	0.73 ± 0.20*+	[0.001,0.02]
F1-Score	0.45 ± 0.10	0.51 ± 0.10*+	[0.02,0.08]

*95 % bootstrap percentile CI for mean difference.

+Bonferroni-adjusted bootstrap percentile CI for mean difference.

Abbreviations: DVH, dose-volume histogram; AUC, area under the curve; AUPRC, area under precision-recall curve; F1-Score, harmonic mean of precision and recall.

3. Results

3.1. Patient characteristics

A total of 179 patients were included in this study. Among them, 27 patients (15.08 %) developed grade 3 or higher radiation esophagitis (RE). Patient characteristics are summarized in Supplemental Table S1. Results from the Fisher’s exact test and the Mann–Whitney U test for categorical, continuous, and DVH variables are also provided in Supplemental Tables S1 and S2.

3.2. Multivariate analysis

The results of all the performance metrics for all models (test set) for predicting RE ≥ Grade 3 are reported in Table 1. Results for the training set are reported in Supplemental Table S3. All multiomic models outperformed the base model. The combined radiomic and dosiomic model achieved the highest AUC and AUPRC. Boxplots comparing AUC and AUPRC for all four model types are also shown in Fig. 2. These results demonstrate that models incorporating omic features outperform the base model, with the combined radiomic and dosiomic model achieving the highest overall performance.

Fig. 2.

Boxplots comparing the performance metrics of all four model types. (A) Area under the curve (AUC) comparison between model types; (B) area under the precision-recall curve (AUPRC) comparison between model types. In both comparisons, most performance metrics for the multiomic models are statistically higher than those of the base model, with the combined radiomic and dosiomic model achieving the highest performance. Statistical significance was assessed using the bootstrap percentile method; a model was considered significantly better than the base if the lower bound of the confidence interval for the mean difference was greater than zero.

The top 10 most important features, ranked by importance, are shown for the base, radiomic, dosiomic, and combined radiomic-dosiomic model types in Fig. 3. This figure illustrates which features appear consistently across 100 models for each model type and shows the average importance of each feature over all 100 iterations. Although the dominant features are highly predictive, the figure also reveals that other features contribute to model performance in a more varied manner, highlighting the value of incorporating a diverse range of features to capture different aspects of the data. For the base model type (Fig. 3A), the relative V60 and V5 metrics are the most important, followed by relative V10, and mean esophagus dose. In the radiomic model type (Fig. 3B), GLRLM High Gray Level Run Emphasis was the most important feature, followed by GLRLM Short Run High Gray Level Emphasis and GLCM Joint Average. In the dosiomic model type (Fig. 3C), first-order Root Mean Squared was the most important covariate, followed by GLCM Inverse Difference Normalized and GLCM Difference Average. Also, in the combined radiomic-dosiomic model type (Fig. 3D), dosiomic Root Mean Squared was the most important covariate, followed by dosiomic first-order Mean, GLCM Inverse Difference Normalized, and others. Descriptions of all features are provided in the Supplemental materials.

Fig. 3.

Most important features for each model type. A, DVH model type top features; B, radiomic model type top features; C, dosiomic model type top features; D, combined radiomic and dosiomic model type top features. The feature importance was determined using the Gini index. Abbreviations: RE, radiation esophagitis; C, CT; P, PET; D, dosiomic; R, radiomic; GLRLM, Gray Level Run Length Matrix; GLCM, Gray Level Co-occurrence Matrix; GLDM, Gray Level Dependence Matrix; GLSZM, Gray Level Size Zone Matrix; Idn, Inverse Difference Normalized.

4. Discussion

In this study, we compared the performance metrics (accuracy, AUC, AUPRC, precision, recall, and F1-score) of the base model with the three omic model types (Table 1) to determine whether multiomic features provided improved predictive value for outcome prediction models (OPMs) of RE over DVH metrics alone. A comparison of the base model with the radiomic, dosiomic, and combined radiomic-dosiomic model types showed that all the multiomic model types outperformed the base DVH model. Additionally, the combined radiomic-dosiomic model type demonstrated the highest performance among the four model types. These results suggest that multiomic features provide valuable information for predicting RE. To the best of our knowledge, this study is the first to comprehensively integrate a diverse array of patient-specific features, including clinical, dosimetric, radiomic (CT as well as PET), and dosiomic characteristics from the esophagus to predict grade ≥3 RE in patients with NSCLC.

Several studies have analyzed radiomic features from CT for predicting RE. A study by Zheng et al. [11] showed that the best-performing models for predicting grade ≥2 RE were the radiomic-only model (AUC = 0.74) and the hybrid model containing clinical, radiomic, and dosiomic features (AUC = 0.75) on the test set. Xie et al. [26] used a combination of radiomics, dosiomics, and deep learning to predict grade ≥2 RE in patients with esophageal cancer undergoing volumetric modulated arc therapy. Their combined radiomic and dosiomic model achieved an AUC of 0.80 on the validation set. Similarly, Ma et al. [25] used dosimetric as well as handcrafted dosiomic features and achieved AUCs of 0.65 and 0.72, respectively, for predicting grade ≥2 RE in NSCLC patients. However, these studies focused on predicting symptomatic RE (grade ≥2). A study by Luna et al. [15], which included 202 NSCLC patients, used clinical and dosimetric features to predict grade ≥3 RE and achieved a maximum AUC of 0.56. Another study by Hawkins et al. [35] incorporated clinical, dosimetric, and pretreatment cytokine levels to predict grade ≥3 RE and reported an AUC of 0.75. Since grade ≥3 RE significantly impacts quality of life—often requiring hospitalization or feeding tube placement—we focus on predicting grade ≥3 RE, as it represents a more clinically significant outcome. Notably, while prior studies have used clinical, dosimetric, or even cytokine data to model grade ≥3 RE, none have utilized radiomic features from both CT and PET imaging in such models. Unlike these prior studies, our model integrates a comprehensive set of clinical, dosimetric, radiomic (from both CT and PET), and dosiomic features to predict grade ≥3 RE in NSCLC patients, an approach that, to our knowledge, has not been previously reported.

Among the four model types that we built, the combined radiomic-dosiomic model demonstrated the highest AUC and AUPRC. Of the top 10 features in this model type, there were three CT radiomic features and seven dosiomic features. The dosiomic first-order Root Mean Squared was identified as the most important. This metric reflects the overall magnitude of the dose distribution by combining both average dose and variability, thereby capturing elevated dose intensities in the esophageal region that may predispose tissue to injury and increase the risk of esophagitis. Other top features in the model included dosiomic first-order Mean, dosiomic GLCM Inverse Difference Normalized, dosiomic first-order Median and CT radiomic GLRLM Short Run High Gray Level Emphasis. As the top feature list included both radiomic and dosiomic features, incorporating such a diverse range of features in the modeling process may support the development of more robust and optimized treatment plans. While no PET-based features appeared among the top 10 most important variables in the combined multiomic model, four PET features were present among the top 20. This suggests that, in our cohort, CT based radiomic and dosiomic features carried the strongest predictive signal for esophagitis risk, while PET features contributed more modestly. This ranking may be due to factors such as the direct anatomical and dosimetric relevance of CT and dose maps, as well as the lower spatial resolution and higher noise typically associated with PET imaging. Nonetheless, the inclusion of PET features within the broader important feature set highlights their potential complementary value in multiomic models.

There are a few limitations to our study. First, this study is a single-institution retrospective analysis, and as a result, external validation of the models could not be performed. We only used the original pretreatment CT, PET, and dose images for feature extraction. However, some studies have examined the utility of various pre-processed (filtered) CT images in addition to the original CT image [36,37]. A future direction for this work could be to evaluate whether features extracted from filtered images enhance model performance beyond what is achieved using features from original images. This is motivated by the study of Demircioglu et al. [38], which suggests that preprocessing filters can influence radiomic analysis and potentially improve model predictive performance. While PyRadiomics generally adheres to IBSI guidelines, it deviates in terms of the binning method. Given the significant variability in the dynamic ranges of the imaging modalities we used (CT and dose images), a fixed bin width approach allows for more meaningful texture comparisons, as the interpretation of gray level differences can vary considerably across images with different intensity distributions. Therefore, we believe that our use of fixed bin width, along with careful preprocessing, supports the IBSI goals of reproducibility and comparability in radiomic and dosiomic feature extraction. Due to the inclusion of many radiomic and dosiomic features in our analysis, we employed a feature reduction technique to avoid overfitting. This may have resulted in the exclusion of features with potential predictive value. However, the data-driven nature of our feature reduction process allowed us to identify strong predictive features. Another limitation of our approach is that, despite correlation-based filtering and the inherent robustness of random forest models to collinearity, the calculated feature importance scores can still be influenced by the presence of highly correlated features. In cases where multiple features capture similar information, the model may split importance across these redundant variables, resulting in lower individual importance scores for each, even if the underlying concept is highly predictive. Consequently, the reported importance for any single correlated feature may underestimate its true predictive value. While this does not diminish the overall predictive performance of the model, it may affect interpretability regarding which specific features are most critical. Future studies may consider alternative feature aggregation strategies or dimensionality reduction techniques to better address this issue. In addition to radiomics and dosiomics, incorporating other omic data such as genomics and proteomics could provide further insights and, as a future direction, these data types could be introduced into the modeling framework to enhance robustness. Developing efficient and effective methods to utilize such multiomic features during the treatment planning phase remains one of the goals of our future research. Moreover, beyond multiomic features, previous studies have shown that incorporating radiosensitivity information–including esophageal expansion [39,40] and esophageal uptake from ¹⁸F-fluorodeoxyglucose positron emission tomography (FDG-PET) [41] may further improve model performance for predicting RE.

In this work, we found that multiomic models are more effective than DVH models in predicting ≥grade 3 RE. The combination of radiomic and dosiomic features achieved the highest AUC and AUPRC, demonstrating that multiomic features extracted from the esophagus enhance RE-based outcome prediction models (OPMs). Future outcome prediction models for radiation esophagitis should integrate both radiomic and dosiomic features to maximize predictive accuracy and enhance clinical decision-making.

Code availability statement

All codes and models associated with this study will be made available on request to the corresponding author.

CRediT authorship contribution statement

Saurabh S. Nair: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing – original draft, Writing – review & editing, Visualization. Ramon M. Salazar: Software, Validation, Writing – review & editing. Ting Xu: Resources, Writing – review & editing. Alexandra O. Leone: Software, Writing – review & editing. Zhongxing Liao: Conceptualization, Writing – review & editing. Laurence E. Court: Conceptualization, Writing – review & editing, Funding acquisition. Joshua S. Niedzielski: Conceptualization, Methodology, Validation, Writing – review & editing, Supervision, Project administration, Funding acquisition.

Funding

Supported by Varian Medical Systems grant to the University of Texas, MD Anderson Cancer Center.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Saurabh Nair, Ramon Salazar, Laurence Court, Joshua Niedzielski, and Alexandra Leone report support through a grant from Varian Medical Systems.

Zhongxing Liao reports RO1 support for this manuscript and consulting fees from AIQ Global, Inc.

Laurence Court also reports grants from the NCI, CPRIT, Wellcome and stock from Leo Cancer Care.

Joshua Niedzielski reports a grant from the Fund for Innovations in Cancer Informatics.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Data availability

Research data is stored in an institutional repository and anonymized data will be shared upon request to the corresponding author once a data transfer agreement has been reached between the requestor’s institution and MD Anderson Cancer Center.