Spatial Radiomic Graphs for Outcome Prediction in Radiation Therapy–treated Head and Neck Squamous Cell Carcinoma Using Pretreatment CT

Abstract

Purpose

To develop a radiomic graph framework, RadGraph, for spatial analysis of pretreatment CT images to improve prediction of local-regional recurrence (LR) and distant metastasis (DM) in head and neck squamous cell carcinoma (HNSCC).

Materials and Methods

This retrospective study included four public pre–radiotherapy treatment CT datasets of patients with HNSCC obtained from The Cancer Imaging Archive (images collected between 2003 and 2018). Computational graphs and graph attention deep learning methods were leveraged to holistically model multiple regions in the head and neck anatomy. Clinical features, including age, sex, and human papillomavirus infection status, were collected for a baseline model. Model performance in predicting LR and DM was evaluated via area under the receiver operating characteristic curve (AUC) and qualitative interpretation of model attention.

Results

A total of 3434 patients (61 years ± 11 [SD], 2774 male) were divided into training (n = 1576), validation (n = 379), and testing (n = 1479) datasets. RadGraph achieved AUCs of up to 0.83 and 0.90 for LR and DM prediction, respectively. RadGraph showed higher performance compared with the clinical baseline (AUCs up to 0.73 for LR prediction and 0.83 for DM prediction) and previously published approaches (AUCs up to 0.81 for LR prediction and 0.87 for DM prediction). Graph attention atlases enabled visualization of regions coinciding with cervical lymph node chains as important for outcome prediction.

Conclusion

RadGraph leveraged information from tumor and nontumor regions to effectively predict LR and DM in a large multi-institutional dataset of patients with radiation therapy–treated HNSCC. Graph attention atlases enabled interpretation of model predictions.

Keywords: CT, Informatics, Neural Networks, Radiation Therapy, Head/Neck, Computer Applications–General (Informatics), Tumor Response, Head and Neck Squamous Cell Carcinoma, Locoregional Recurrence, Radiotherapy, Deep Learning, Radiomics

Supplemental material is available for this article.

© RSNA, 2025

Summary

Radiomic graph analysis of pretreatment CT enables holistic analysis of head and neck squamous cell carcinoma tumors and their surrounding environment, showing good performance in local-regional recurrence and distant metastasis prediction and unique model interpretability.

Key Points

• ■ In a multi-institutional study of 3434 patients with head and neck squamous cell carcinoma, spatial modeling of pre–radiotherapy CT images via radiomic graph analysis (RadGraph) resulted in area under the receiver operating characteristic curve (AUC) values up to 0.83 and 0.90 for local-regional recurrence (LR) and distant metastasis (DM) prediction, respectively.

• ■ RadGraph showed higher performance compared with baseline methods, including previously published models and predictive clinical features (AUC for clinical baseline: up to 0.73 for LR prediction and 0.83 for DM prediction).

• ■ RadGraph analysis enabled interpretation of machine learning predictions, identifying which regions within the head and neck anatomy most greatly impacted model performance.

Introduction

Head and neck squamous cell carcinoma (HNSCC) accounts for approximately 4.5% of cancer diagnoses and deaths worldwide (1). HNSCC is commonly and effectively treated with radiation therapy (RT) with or without chemotherapy. However, a subset of patients with HNSCC will experience treatment failure of either local-regional recurrence (LR) or distant metastasis (DM) at a rate of approximately 15% and 10%, respectively (2). Each of these can result in higher rates of mortality and disease progression. The ability to predict which patients are at greater risk of LR and DM may be able to inform decisions regarding treatment and posttreatment surveillance, potentially enabling personalized patient care and improved patient outcomes.

While the exact cause of LR and DM following RT is not completely understood, previous research has implicated various predictive factors for HNSCC treatment failure (3–5). For instance, nodal invasion of HNSCC has been shown to be strongly associated with tumor aggression (5). In this study, we seek to determine whether the interplay between tumors and their surrounding environments can be modeled in medical imaging to predict treatment outcomes for RT-treated HNSCC.

Radiomic analysis of medical images has been shown to be valuable in diagnosing and predicting outcomes for many diseases, including HNSCC (6–13). Commonly, these approaches entail the extraction of a collection of numerical values, each representing one radiomic feature of a region of interest (ROI) within an image. Most medical imaging studies are limited to radiomic analysis of a single ROI, such as the tumor (8–10). Those studies that have investigated radiomic features from peritumoral ROIs largely adopt simplistic feature concatenation schemas or study the peritumoral and tumor ROIs independently, failing to capture radiomic feature interaction between tumor and extratumoral compartments (11–13).

Previous machine learning studies have attempted to predict both LR and DM for HNSCC from pretreatment CT imaging, with varying degrees of success (9,11–15). Vallieres et al (9) studied three-dimensional radiomic features extracted from the gross target volume (GTV) of patients with RT-treated HNSCC (n = 300) on both CT and PET images and were able to predict DM but not LR successfully. In a follow-up study, Diamant et al (14) designed a custom convolutional neural network (CNN) to predict LR and DM from pretreatment CT images from the same dataset, improving performance. Mateus et al (15) further improved the architecture of the CNN introduced by Diamant et al and demonstrated that their new model could generalize results to an unseen test dataset, studying a total of 435 patients.

These studies utilized imaging features extracted solely from the tumor (GTV) rather than from surrounding peritumoral regions, limiting both their interpretability and ability to capture any information that might be harbored in the tumor microenvironment and surrounding patient anatomy. Furthermore, these studies reported variability in whether imaging-based features were able to effectively predict treatment outcomes at all. Additionally, none of these studies were validated in a multi-institutional dataset larger than 500 patients.

Because HNSCC outcomes are strongly dependent on pathology beyond the bounds of just the tumor, we believe that previous tumor-centric radiomic models fail to analyze valuable information that might improve prediction of both LR and DM from imaging signatures outside of the tumor. Here, we propose a radiomic graph analysis framework called RadGraph, leveraging computational graphs and graph attention networks (GATs) (16) to synthesize information from multiple regions within a CT scan to predict LR and DM for RT-treated HNSCC. Our proposed method leverages radiomic information from within the tumor as well as from neighboring anatomic regions to improve the prediction of treatment failure, modeling the interaction between features in each region rather than analyzing each subset of features independently. Our approach is uniquely interpretable, providing insight into which anatomic regions are most influential in predicting treatment failure. We evaluate our approach on a multi-institutional dataset of 3434 patients, larger than any previous work studying CT-based prediction of LR and DM in HNSCC.

Materials and Methods

Study Sample

In this retrospective, Health Insurance Portability and Accountability Act–compliant study, data from 3434 patients with HNSCC from the Computed Tomography Images from Large Head and Neck Cohort (RADCURE) (17,18), Head-Neck-PET-CT (HNPET) (9,19), HNSCC (MD Anderson Cancer Center, MDACC) (20–22), and Head-Neck-Radiomics-HN1 (HN1) (8,23) collections publicly available from The Cancer Imaging Archive (24) were analyzed (Fig 1). Because all cases were de-identified and obtained from The Cancer Imaging Archive, institutional review board approval was not required for this study. Patients were excluded due to having insufficient follow-up data for the studied outcomes, no tumor segmentation, or a lack of imaging data.

Figure 1:

Patient inclusion and exclusion criteria. Flowchart shows the patient inclusion data for the four TCIA datasets studied, as well as the composition of the training, validation, and testing datasets studied in this work. GTV = gross target volume, HNPET = Head-Neck-PET-CT dataset, HN1 = Head-Neck-Radiomics-HN1 dataset, MDACC = MD Anderson Cancer Center head and neck squamous cell carcinoma dataset, RADCURE = Computed Tomography Images from Large Head and Neck Cohort dataset, TCIA = The Cancer Imaging Archive.

CT Images

For each patient, pretreatment CT scans and associated GTV contours from RT structure files were obtained. CT acquisition parameters are listed in Table S1.

Study Outcomes

The outcomes studied in this work were binary occurrence of LR and DM following RT for HNSCC. Previous studies have performed these binary classification tasks without setting a minimum follow-up threshold to accommodate small dataset sizes. Because our dataset includes substantially more patients than these studies, we have adopted a 2-year minimum follow-up requirement as described in both Mateus et al (15) and Kazmierski et al (18). This decision was made both to fit the binary classification task schema of our approach, as well as to ensure that a minimum amount of follow-up was performed for each patient. Additionally, this threshold is shared with previous studies, enabling a direct comparison of methods and results. Finally, this follow-up requirement is in accordance with studies finding that the majority of treatment failure events occur within 2 years of treatment (15,25).

Radiomic Graph Analysis

The entire pipeline for radiomic graph analysis is described in Figure 2.

Figure 2:

Study overview shows the radiomic graph framework (RadGraph), as well as its potential place in the current radiation treatment pipeline. RT = radiation therapy, 3D = three-dimensional.

Supervoxel identification and radiomic feature extraction.

For each CT scan, a three-dimensional region encompassing 5 cm beyond the bounds of the GTV contour was identified (Fig 2). The zero-parameter Simple Linear Iterative Clustering (26) algorithm was used to identify approximately 100 supervoxels within this region. Three-dimensional radiomic features were extracted from within each of these supervoxels in addition to the GTV region, using the PyRadiomics (27) software package. A total of 93 features were extracted for each supervoxel from each patient CT scan. Radiomic feature expression from within the GTV was used to identify top predictive features for each studied outcome using the minimum redundancy maximum relevance (28) algorithm in the training set. Among these selected features, the best combination of features was determined by random forest classification performance on the held-out validation set. This process mirrors that of common radiomic analysis pipelines and formed the basis of the traditional radiomics baseline approach presented in the results. Selected radiomic features for all supervoxels were then normalized with respect to minimum and maximum GTV radiomic feature expression.

Radiomic node and edge creation.

The top 20 non-GTV supervoxels with the most similar radiomic feature expression to the GTV were identified by calculating the Euclidean distance between feature values of each supervoxel and the GTV. From these 20 selected supervoxels, 20 nodes were created, with an additional 21st node representing the GTV. Edges were created between each non-GTV node and the GTV node to capture predictive information available in non-GTV regions. This resulted in 3434 radiomic graphs.

GAT modeling.

To model these graph representations, a GAT was used to predict the binary occurrence of LR and DM for each patient. GATs make use of an attention mechanism to iteratively update each node’s features using information from neighboring nodes. Further details regarding our GAT architecture and hyperparameters used for model training can be found in Table S2. All hyperparameter values were determined based upon model performance on the validation set to avoid overfitting on held-out test sets. Due to the structure of our radiomic graphs containing a central node representing the GTV connected via one edge to each of the non-GTV nodes, we altered the “vanilla” GAT structure to include a final “read-out” layer composed of updated node features from the GTV node rather than an aggregate of all node features for the binary classification task. This decision followed from previously published studies in which analysis of radiomic features from the GTV alone were predictive of patient outcomes following RT; our method expands upon these approaches by updating GTV radiomic features using radiomic features from surrounding, nontumor regions prior to LR and DM prediction. This is a unique feature of RadGraph facilitated by the use of GAT analysis.

Graph attention atlas creation.

Another advantage of using GAT modeling in RadGraph is the ability to interpret our models via the creation of “graph attention atlases.” Using the HN1 test dataset, all patient CT images were coregistered using a previously studied configuration (29) of the Elastix (30) platform with both rigid and B-spline registration. In this manner, all patients from the dataset were coregistered to a single reference patient so that model attention maps could be directly overlayed upon one another. To obtain these attention maps, attention values from the GTV readout node to all other graph nodes were extracted from the final GAT layer of each model studied. These attention values were then matched to corresponding CT supervoxels for each patient. By overlaying these attention values across registered patient CT images stratified by tumor subsite, graph attention atlases were produced for oropharyngeal and laryngeal tumors. This enabled the visualization of regions within the head and neck area most significant in model prediction of LR and DM for these patients.

Lymph node attention analysis.

Additionally, lymph node GTV (GTVn) contours were available for a subset of patients (n = 109) within the HNPET test dataset corresponding to nodal volumes targeted by radiation. Because not all patients had such targets, these GTVn contours were not included as input into GAT models, nor were they used as predefined ROIs for radiomic feature extraction. However, by calculating supervoxel attention following model training, we were able to identify when high areas of model attention corresponded with GTVn contours in a retrospective analysis.

Evaluation framework.

To evaluate the generalizability of our models, we trained, validated, and tested our models on multiple independent datasets. For the RADCURE dataset, we adopted a similar data split to that of the original article, using the same training dataset (n = 1576) and the same test set (n = 532) and the remaining 379 patients not studied in the original article as a validation set (18). HNPET (n = 288), MDACC (n = 535), and HN1 (n = 124) were all used as independent test sets. Performance on the validation set was used to determine all hyperparameter settings for both RadGraph and baseline experiments to avoid overfitting to the test datasets using the WandB (31) software (version 0.15.10; Weights & Biases) package.

Statistical Analysis

Models were evaluated using the area under the receiver operating characteristic curve (AUC). Comparisons were made with previously published imaging- and clinical feature–based models (details concerning the implementation of these approaches can be found in Appendix S1). A clinical baseline was also studied, consisting of a random forest machine learning model trained solely on clinical variables. These clinical factors included whether a patient underwent concurrent chemoradiation therapy, in addition to patient human papillomavirus infection status, sex, age, tumor stage (American Joint Committee on Cancer seventh edition), Eastern Cooperative Oncology Group performance status, tumor subsite (oropharyngeal, laryngeal, etc), and tumor volume (assessed using dimensions of GTV contours). The combination of these features with RadGraph was also explored by concatenating clinical features to the input of the final layer of the RadGraph GAT network. Statistical significance in model performance differences was not assessed.

Results

Patient Characteristics

A summary of patient data is provided in Figure 1 and the Table. A total of 3434 patients were studied. RADCURE contained 2487 patients with a mean age of 62.03 years ± 11.65 (SD) and 489 (19.7%) female patients. HNPET contained 288 patients with a mean age of 63.20 years ± 10.19 and 70 (24.3%) female patients. MDACC contained 535 patients with a mean age of 57.90 years ± 9.25 and 78 (14.6%) female patients. HN1 contained 124 patients with a mean age of 61.90 years ± 8.86 and 23 (18.5%) female patients.

Patient Characteristics

Performance of RadGraph in Predicting LR

A total of four radiomic features measuring the gray-level size-zone matrix and gray-level co-occurrence matrix were found to be most predictive of LR based upon feature selection experiments (Tables S3, S4). Using only these features from CT imaging as input, RadGraph modeling resulted in AUCs between 0.57 and 0.66 on the four test datasets studied (Fig 3). The addition of clinical features to RadGraph resulted in AUCs of 0.60–0.83 across the four studied datasets, achieving the highest performance of all methods studied in all datasets but the MDACC test set. The sensitivity and specificity values for all studied models for LR prediction are provided in Table S5.

Figure 3:

Model performance on outcome prediction tasks. Graphs show AUCs for RadGraph and baseline model predictions of LR (top) and DM (bottom) across the four datasets studied. Bars outlined in dashed red lines indicate the highest performance achieved on each dataset. AUC = area under the receiver operating characteristic curve, CNN = convolutional neural network, DM = distant metastasis, HNPET = Head-Neck-PET-CT dataset, HN1 = Head-Neck-Radiomics dataset, LR = local-regional recurrence, MDACC = MD Anderson Cancer Center Head and Neck Squamous Cell Carcinoma dataset, RADCURE = Computed Tomography Images from Head and Neck Cohort dataset, RadGraph = radiomic graph framework.

Performance of RadGraph in Predicting DM

Six radiomic features measuring original imaging intensity, the neighboring gray-tone difference matrix, the gray-level difference matrix, the gray-level co-occurrence matrix, and the gray-level size-zone matrix were found to be predictive of DM from feature selection experiments (Table S3). When trained on these CT-based radiomic features, RadGraph achieved an AUC of 0.62–0.90 for the DM prediction task across the four test datasets studied (Fig 3). The addition of clinical features to RadGraph resulted in AUCs between 0.64 and 0.86. Again, RadGraph had the highest performance of all methods studied in all datasets except for the MDACC test set. The sensitivity and specificity values for all studied models for DM prediction are provided in Table S6.

Visualization of High-Attention Regions

Attention values were extracted from trained GAT models for LR and DM and used to create graph attention atlases for the HN1 dataset (Fig 4) and to quantify model attention to GTVn regions for the HNPET dataset (Fig 5).

Figure 4:

Graph attention atlases created via coregistration of patient CT images and attention maps from the HN1 dataset for oropharyngeal and laryngeal HNSCC tumors. Images in a 56-year-old male patient are shown. Top: Atlases for attention values from models predict local-regional recurrence. Bottom: Atlases for models predict distant metastasis. HN1 = Head-Neck-Radiomics-HN1 dataset, HNSCC = head and neck squamous cell carcinoma.

Figure 5:

Radiomic attention to GTVn. Bar graphs show model attention values to GTVn regions from the HNPET dataset for models predicting local-regional recurrence (top) and distant metastasis (bottom). Model attention is calculated directly from the final GAT layer and discretized into low, medium, and high values. GAT = graph attention network, GTVn = lymph node gross target volume, HNPET = Head-Neck-PET-CT dataset.

Graph attention atlases revealed high model attention to peripheral regions for LR prediction, in proximity to cervical lymph node chains. For DM prediction, model attention appeared to be highest just lateral to the cervical vertebrae. For oropharyngeal tumors, model attention scores were generally lower for DM prediction, with few areas of high attention.

Within the HNPET dataset, GAT models for both LR and DM prediction displayed high attention to GTVn regions (Fig 5) despite these contours not being used as input for model training or evaluation.

Discussion

We found that the use of our RadGraph framework was able to effectively predict treatment failure for RT-treated HNSCC in a multi-institutional dataset of 3434 patients with pretreatment CT scans. When combined with clinical features, RadGraph achieved AUCs of up to 0.83 and 0.90 for LR and DM prediction, respectively. Furthermore, our trained models were interpretable in a post hoc manner, providing insight into which regions of the head and neck anatomy contributed to model outputs.

Previous works have reported varying results when attempting to model LR and DM from pretreatment CT scans. We investigated four methods representative of those proposed in the literature, including a traditional radiomic approach, two previously published CNN models, and a clinical baseline approach. Studies using these methods had been limited to datasets not larger than 500 to study LR and DM, and, to our knowledge, our work represents the first to evaluate them in a large, multi-institutional dataset. Compared with these previous methods, RadGraph leveraged two key technical innovations. First, computational graphs were used to represent the relationship between tumors and their anatomic surroundings, enabling a more complete analysis of CT imaging data. Second, GATs were used to model these graphs, allowing us to quantify the relationship between radiomic feature expression within different regions of the tumor ecosystem. The use of GATs also provided an avenue for model interpretation. Our graph radiomic approach showed higher performance compared with all other methods on three of four datasets studied. On MDACC, all methods, including RadGraph, performed less effectively than on all other datasets. This difference in model performances across different datasets reflects previous study findings (18) and might be investigated in future studies.

RadGraph also provides a unique method for model interpretability. While methods for CNN interpretability do exist, approaches in the study of HNSCC (14,15) have generally analyzed solely GTV ROIs and have not provided meaningful analysis into model decision-making. By visualizing the distribution of GAT attention for RadGraph models, we were able to identify which regions of the head and neck anatomy were most significant for LR and DM prediction. Graph attention atlases revealed that attention was concentrated near cervical lymph node chains when predicting LR, consistent with our finding of high attention to GTVn regions in the HNPET dataset. For DM prediction, graph attention atlases appeared to demonstrate higher attention in the posterior neck just lateral to cervical vertebrae. Future work is required to comprehensively understand these qualitative findings, but we believe that the ability to visualize model attention is a unique strength of RadGraph that might build trust and understanding in radiomic-based predictions of patient outcomes. Additionally, for both LR and DM prediction, we found that RadGraph models showed high attention to GTVn regions in the HNPET test dataset. Because GTVn structures were not used as input for model training, we believe this retrospective analysis is an indication that our GAT models were able to identify disease-relevant ROIs autonomously and use information from these regions for LR and DM prediction. This is one strength of our GAT framework that contrasts with other machine learning approaches that might only take the GTV as input.

There were several limitations to our study. First, while our model did enable the identification of radiomic features of significance in addition to imaging regions of interest, it did not provide insight into which clinical features were most significant in predicting LR and DM. While this weakness is shared with the other methods studied, future work might improve model performance by modeling only those clinical features most important to predicting treatment failure. Furthermore, this study employed a primarily unsupervised approach in identifying image supervoxels and, subsequently, radiomic graph nodes. This was a constraint of the available data rather than our method itself, which could be adapted to include predefined supervoxels of interest. Future work might model physician-defined ROIs to direct model attention based on medical expertise. Additionally, our study modeled binary prediction using 2-year cutoffs for the outcomes of interest, in line with previous works. While we believe this choice improved upon approaches that do not require a minimum follow-up for binary classification, future work can improve upon our method by incorporating time-to-event analysis. Finally, the dataset studied was composed of images acquired with numerous different acquisition parameters and settings (Table S1). While this dataset was representative of real-world heterogeneity, and our work suggests that RadGraph is able to generalize across these settings, future work might further investigate the relationship between these parameters and model performance.

In conclusion, this study is the first, to our knowledge, to evaluate the use of CT-based radiomics in predicting LR and DM in a large multi-institutional dataset (n = 3434). Because all of the studied data are publicly available on The Cancer Imaging Archive, we believe that the proposed method and baselines studied can serve as a benchmark for future comparison. Our approach showed higher performance in LR and DM prediction compared to other published methods, generalizing well across multiple public datasets. We believe that this generalizability, in addition to the ability to visualize model attention, demonstrates the robustness of our method and its potential for clinical impact in RT-treated HNSCC. Future studies are needed to validate our findings. A radiomic-based model might prompt personalized treatment regimens, such as use of alternative or more aggressive treatment modalities or more frequent posttreatment surveillance schedules for patients with HNSCC found to be at high risk of LR or DM based upon pretreatment imaging. Furthermore, our RadGraph approach may be applied to other tasks, potentially providing interpretable and accurate models for other medical imaging problems.