Artificial Intelligence and Radiomics in Head and Neck Cancer Care: Opportunities, Mechanics, and Challenges

Abstract

The advent of large-scale high-performance computing has allowed the development of machine-learning techniques in oncologic applications. Among these, there has been substantial growth in radiomics (machine-learning texture analysis of images) and artificial intelligence (which uses deep-learning techniques for “learning algorithms”); however, clinical implementation has yet to be realized at scale. To improve implementation, opportunities, mechanics, and challenges, models of imaging-enabled artificial intelligence approaches need to be understood by clinicians who make the treatment decisions. This article aims to convey the basic conceptual premises of radiomics and artificial intelligence using head and neck cancer as a use case. This educational overview focuses on approaches for head and neck oncology imaging, detailing current research efforts and challenges to implementation.

INTRODUCTION: BEATING THE HUMAN EYE OR BRAIN

Radiomics and artificial intelligence (AI) have become buzzwords in the oncology research field and are now often the go-to proposed solution for complex clinical prediction problems when imaging is available. This is fueled by the promise of radiomics (machine-learning approaches for image features analyses) and AI (neural networks that learn from the image directly) to be able to extract information in images that cannot be observed by the human eye.¹ Nevertheless, more than just detecting meaningful features that cannot visually be observed, these machine-learning pipelines can facilitate pattern recognition in images, detection in biomarker data, and integration with nonimaging variables that cannot be comprehended by the human brain, due to the large number of potential variables.^2–4

This article aims to convey the basics of radiomics and AI in relation to head and neck oncologic imaging by addressing the application space for leveraging medical images, radiomics, and basic deep-learning mechanics and the challenges of cancer imaging– enabled AI specifically for patients with head and neck cancer.

OPPORTUNITIES: MODEL-BASED PERSONALIZED TREATMENT DECISION-MAKING

In general, medical machine learning can be categorized as focused on regression or classification tasks, which depends on the predicted outcome. Regression methods (e.g., linear regression, polynomial, and support vector regression) aim to predict continuous values, such as tumor volume reduction, weight loss, or biomarker values, whereas classification methods (e.g., logistic regression and random forest) aim to predict a binary/categorical value, such as tumor recurrence or toxicity development. An additional deviation of the classification is Cox hazard regression, which incorporates the time-to-event together with a binary event (e.g., death or censored). Using these classification and regression approaches, a broad swath of problems can be addressed that, on the surface, seem radically distinct in scope. For example, similar deep-learning approaches can be used for tasks as different as risk stratification, biomarker detection, image segmentation, and image reconstruction.

The most evident example is risk prediction of overall survival or tumor control after treatment based on imaging tumor and clinical characteristics in patients with head and neck cancer. By identifying patients who are at specific high or low risk of treatment failure before treatment, therapy can be tailored to their anticipated risk. For instance, HPV-positive tumors have exceptionally good outcome rates (5-year overall survival approximately 80%^5–7); thus, having provoked clinical trials to reduce radiation fractionation dose,⁸ magnetic resonance (MR)–guided adaptive radiation dose de-escalation,⁹ and sparked discussions about the relative, as-yet-unquantified, potential benefits associated with chemotherapy for these patients.^10,11 Nevertheless, not all patients with HPV-associated oropharyngeal cancer may do well with a de-escalated treatment regimen; thus, models based on radiomics combined with clinical variables (e.g., smoking status) and biomarkers (e.g., HPV subtypes) can play a role in directing treatment alterations more effectively. Similarly, for patients with head and neck cancer at high risk of treatment failure, clinical trials can target these patients specifically for intensified treatment regimens (e.g., image-guided doseescalation or induction chemotherapy). Although intensified regimens may not be effective when applied to the full population with head and neck cancer and may do more harm by increasing treatment-induced toxicity rates, it could be effective in a cohort saturated with high-risk patients only. Moreover, in general, for any treatment approach in which variable effectiveness response can be observed, such as immunotherapy for patients with head and neck cancer,^12,13 prediction models can be potentially developed to identify patients who are likely to experience treatment success versus treatment failure and alter treatment accordingly.

Another application for image biomarkers is toxicity development after chemoradiation, as it is highly variable in patients with head and neck cancer (i.e., patients who receive a similar dose to critical organs can have a completely different toxicity profile after treatment). Tumor information is readily available not only in the form of clinical CT, PET, and MRI but also that of the normal tissues. Normal tissue image features can be extracted and may provide patient-specific radiosensitivity characteristics that are associated with toxicity development. For instance, the prediction of late xerostomia was improved with parotid gland CT radiomics features,¹⁴ which indicated that higher heterogeneity of the parotid glands was associated with a higher risk of xerostomia. Subsequently, from T1-weighted MRI image biomarkers, it was observed that higher fat concentrations in the parotid gland were associated with a higher risk of developing xerostomia.¹⁵ A handful of other studies have investigated the relationship between CT information and the development of a radiation-induced toxicity,^16–18 whereas the use of MRI information on normal tissues is a relatively uncharted domain.^19,20

Other areas within head and neck oncology imaging in which AI has already had a practical impact is autosegmentation of tumor or organ at risk, which is, in essence, a voxel-based classification task.²¹ The clinical usability of autosegmentation is to aid radiation oncologists (or clinicians) in organ and tumor definition, decreasing delineation times.^22–25 Furthermore, AI is being deployed for MRI deformation,^26–28 improved image reconstruction,^29–31 and MRI to pseudo-CT.^32–35 These applications show particular promise for MRI-based adaptive radiation and combined imaging-treatment device (such as MRI-linear accelerator) applications, as improved deformation can improve adaptive warping of contours to new scans. Adequate pseudo-CT approximations will allow for dose optimization from MRI only, whereas today, a CT scan is required to calculate radiation absorption for treatment planning.

This capacity to generate usable data is not limited to therapy delivery, however. For example, AI is being applied to develop MR fingerprinting, which is an acquisition strategy allowing multiparametric MRI reconstruction from a single acquisition.^36,37 The practical benefit of MR fingerprinting is that it allows for the acquisition of multiple quantitative MR sequences (such as T1 or T2 maps) from a single scan acquisition, significantly reducing the total scan time. Because this process involves deep learning to reconstruct the images, the same deep-learning approach could simultaneously allow front-end integration of radiomics, segmentation, and motion correction in a “near live” time interval. This idea of solving multiple algorithmic processes simultaneously is an especially exciting developmental opportunity for AI-based approaches (but necessarily requires sufficiently large and representative training data sets).

Although radiomics and AI research studies are emerging exponentially, clinical implementation is often not (yet) reached for most of the prediction models at a level of clinical usability or with interpretable visualization interfaces. To improve implementation, models must be presented in a manner in which the results can be understood and adequately interpreted by clinicians, whom are responsible for making the treatment decisions. Moreover, the models must be reliable (i.e., well trained and validated on representative data), as well as transparently inform on risk uncertainty per individual prediction. This need for decision-support tools that allow for automatic integration of data processing, prediction, and quality assurance with effective interpretability to the end user is an imperative. The next sections will deal with the mechanics of radiomics and AI in head and neck cancer to improve the understanding of their potential and challenges.

MECHANICS: FROM PICTURE APPRECIATION TO ARTIFICIAL INTERPRETATION

Current oncology practice is necessarily dependent upon medical images, and this dependence has increased in the last decades. CT scans are generally considered the standard imaging modality for diagnostic and therapy planning purposes, with their intensities consistently representing electron tissue density. For patients with head and neck cancer in particular, MRI is a valuable additional image modality³⁸ due to its exceptional soft tissue contrast as a result of the decomposition of radiofrequency resonance. Additionally, by detecting metabolic activity of tissues, PET can provide physiologic information to aid in distinguishing between normal and tumor tissues. This explains why MRI and PET are generally regarded as indispensable in tumor diagnosis, delineation, and response assessment,^39,40 as well as detection and tumor border definitions.^41–43 Moreover, the introduction of functional MRI sequences, such as diffusion-weighted imaging and dynamic contrast enhanced, provide spatially localized physiologic information in frame with anatomy, introducing an additional dimensionality to differentiate tumor from normal tissue.^44–51

Semantic Image Evaluation

Because a trained human eye is excellent in pattern recognition, qualitative image evaluation by the radiologist has historically been the conventional clinical standard. Consequently, initial imaging research focused on semantic image features, which are image content descriptions according to human perception (e.g., “radiologically positive lymph node,” “hypodense tumor,” or “heterogeneous lesion” on a radiologist report), or simple image characteristic values (e.g., maximum longitudinal tumor diameter and mean intensity values). For instance, simple MRI and ¹⁸F-fluorodeoxyglucose PET evaluation measures improved the diagnostic accuracy for malignant lymph node detection.⁵² The predictive value of semantic imaging features directly translates to the clinical TNM staging based on radiologic image evaluation.⁵³ Furthermore, Wang et al⁵⁴ showed that tumor invasion and heterogeneity evaluation on MRI were predictive for local control in patients with nasopharyngeal cancer.

The improved quality and quantity of medical imaging of the last decades have opened doors to move from semantic or qualitative observer assessments (primary classification tasks) to quantitative machine-readable image measures for model-based treatment decision support. Additionally, the current climate of computational processing power, available large-scale (imaging) data, machine-learning, and AI approaches are now reaching a stage to exploit the evergrowing imaging data.

Machine Learning

In the early 2000s, terms such as computer-aided treatment, computer-aided detection, or computer-aided diagnosis were used to describe computational processes that process clinically useful image information.⁵⁵ In the last decades, AI has made is entrance in the head and neck oncology field. The definition of AI is the capability of a machine to imitate intelligent human behavior and is a collective term for all computer processes that fall under this definition.

Machine learning is a subfield within AI (Fig. 1), which in a broader sense refers to a computerized algorithm that learns to solve tasks by recognizing patterns in the data.⁵⁶ In other words, the “machine” (i.e., statistical model) learns from the associations in the presented data by minimizing the model’s cost function. Predefined variables can be continuous (e.g., height and weight), categorical (smoking status: never, former, or current), ordinal (e.g., T-stage and performance score), or binary (yes or no; 1 or 0). Machine learning for medical images requires feature extraction from specific areas in the images, such as mean intensity or volume of the delineated tumor. More advanced imageprocessing techniques have allowed for image characteristic extraction that can quantify more comprehensive and complex aspects of an image, referring to the umbrella terminology radiomics.

FIGURE 1.

Definitions in Artificial Intelligence

RADIOMICS: IMAGE FEATURE MINING

The wealth of raw images acquired for oncology treatment is a great source of information for patient-specific anatomy and physiology and can be used beyond the scope of tumor detection and delineation. Radiomics refers to the process of quantifying patient-specific tissue characteristics from images into features (i.e., image biomarkers), translating intensity, texture, and geometric properties of the tissue from a specific volume of interest into single variables that can be fed into a machine-learning modeling process (Fig. 2). Although these features can describe similar characteristics as semantic features (think: heterogeneity of the tumor), the hypothesis is that they can reveal more information in a quantitative manner, because the medical images are processed as data not as pictures.^57,58 Although image texture analyses originated in the 1970s to 1980s,^59–61 the radiomics concept was not introduced until 2012 by Aerts et al,⁴ proposing comprehensive extraction of geometric, first-order, and texture feature extraction, thereby converting medical images into high-dimensional minable data.^3,58,62 Aerts et al⁴ showed that pretreatment CT-based radiomics features, describing the density, heterogeneity, and shape of the tumor, could predict overall survival of patients with non–small cell lung cancer and validated this in an independent patient cohort of both head and neck and lung cancer. Several subsequent studies have confirmed that tumor radiomics features can contribute to the prediction of overall, disease-free, and progression-free survival in patients with head and neck,^4,63–66 lung,^4,67 cervical,⁶⁸ prostate, and rectal⁶⁹ cancer. The explanation of radiomics features is often intuitive and relatable to the semantics, such that high values of texture feature that capture higher heterogeneity of the tumor are associated with a higher risk of recurrence.^57,66 The geometric features can indicate that larger (e.g., volume) and more irregularly shaped (e.g., surface and bounding box) tumors are related to worse survival rates.⁶⁶ Radiomics features have been shown to have additional prediction value to clinical variables such as age, TNM stage, and performance score, but they can also replace them. As shown by Zhai et al,⁶⁶ the feature major axis length of all lymph nodes (i.e., the maximum length between pathologic lymph nodes) performed better in predicting survival than N stage. This indicates the potential of quantification of clinical variables, as well as the importance of being wary of multicollinearity (i.e., high interassociations between the independent variables).

FIGURE 2. Radiomics Extraction Process.

The segmented volume of interest is extracted from an image (I). The following radiomics types are extracted from this volume of interest: intensity image intensity features (II), geometric features (III), and texture features (IV). A gray-level run length matrix is given as example for texture calculation, which quantifies the number of repetitions of gray intensities from left to right.

Abbreviations: 3D, three-dimensional; Max, maximum.

DEEP LEARNING: MODELING DIRECTLY FROM THE IMAGE

Deep learning is a subtype of machine learning (Fig. 1) yet distinguishes itself from general machine learning by not requiring predefined features or variables, but rather inculcating both feature selection and variable weighting simultaneously directly from the image or data within the deep-learning network.^70,71 In imaging, the first layers of the neural network are convolutional layers, which are different sequential combinations of imaging filters that emphasize or suppress certain textures or patterns (Fig. 3). The convolution and fully connected network layers can extract image characteristics (e.g., eyes, tumor heterogeneity, and organ at-risk edges) without the human engineering of variables, thus being autodidactic in determining the most important aspect of the images. Similar to other machine-learning techniques, but even more prominent for deep learning, is the need to be wary for overfitting and lack of generalizability of the deep-learning model to new data. Important algorithm implementations have been made to charge a network for overfitting (e.g., dropout, internal cross validation, and permutations); nevertheless, it is extremely important to have a completely independent validation test set that has not been used during any part of model training process.

FIGURE 3. Deep Convolutional Neural Network Example for Classification Prediction Problem.

Starting with an MRI, for example, as the input layer, followed by convolutional, pooling, and flatten layers, which is followed by fully connected layers to output the endpoint local tumor control vs. local tumor failure.

Abbreviation: Max, maximum.

CHALLENGES IN MODELING

As for any statistical model, an overarching challenge is generating a generalizable model, meaning that the model will perform well in new unseen data. Key challenges that jeopardize the model generalizability and fragility are (1) the risk of false-positive associations, (2) over- and underfitting, (3) data representation bias, (4) unbalanced data, (5) multicollinearity, and (6) model result interpretability.

A false-positive variable refers to a statistically significant variable (p value less than set threshold) without a real underlying association of a variable with the outcome.⁷² This is particularly important for radiomics features, as they may be highly coherent in the representation of an imaging characteristic of a given region of interest. Radiomics approaches specifically would be well advised to carefully implement correction strategies as a function of the number of candidate biomarkers/thresholds/features. A proposed approach is multiple testing corrections,⁷³ but they are often considered too simplistic for machine-learning approaches, and there is limited consensus on p-value adaptation in machine learning, as other methods may be more appropriate. This is a less direct issue for deep learning, but a network can detect “significant” areas in an image that for a biased relation with the endpoint or creating false areas in case of reconstruction.⁷⁴

Overfitting or underfitting are main concerns in machine learning.⁷⁵ Although underfitting refers to not giving the model enough degrees of freedom to fit to the data (Fig. 4, left), overfitting refers to fitting a model very specifically on training data (Fig. 4, right). In both scenarios, this results in a model that does not perform well when presented to new data. In other words, the model is not “generalizable” to external data. Addressing under- and overfitting remains a challenge during all steps of any machine-learning analysis that requires serious and continuous considerations.

FIGURE 4. Example Case for a Classification Task.

To distinguish the patients in red (e.g., recurrences) from those in black (e.g., local tumor control), a model is optimized to define a borderline that increases the distance between the two groups. Examples of model fits that are underfitted (left), adequately fitted (middle), and overfitted (right) are depicted.

The patient sample size can cause incorrect model fitting and, hence, make a model ungeneralizable to new data. The main concern of training a model on a small data set is that it is not an accurate representation of the full population. Moreover, medical data are often noisy, contain outliers, and nearly always hold some incorrect or erroneous information, due to clinical complexity and human error. Moreover, processes and reactions within the body are rarely fully predicted with a handful of variables in a simple linear manner. In other words, adequate patient numbers are required to represent medical processes and predict clinical outcome measures.

Unbalanced data can be a concern even in a representative cohort, as the dependent variable can be imbalanced, referring to a low number of events in the data set. For example, predicting the local recurrence with tumor image features in a cohort of patients with HPV-positive stage I to II oropharynx carcinomas is problematic due to extremely low local recurrence rates (approximately 2 in 100 patients). Fig. 5 demonstrates that the area under the curve performance measure is likely to be unrealistically high in unbalanced data, as isolated events can be falsely be detected. To mitigate this, the number of patients and events need to be increased. The pragmatic rule of thumb is that per 10 events in a data set, a model can contain a variable. Nevertheless, this is just a guideline and should be carefully evaluated in combination with overfitting monitoring measures.⁷⁶ For deep learning, upsampling methods have been proposed to overcome the shortcoming of imbalanced data; however, similar to general machine learning, the training performance measures must be evaluated with caution, and effectiveness needs to be better investigated⁷⁷; regardless, the model will always need to be tested on a large enough imbalanced data set.

FIGURE 5. Logistic Regression Area Under the Curve Performance in Random Data for Highly Imbalanced Event Data (2% Event Rate, Left) and More Balanced Data (30% Event, Right).

AUC for imbalanced data is unrealistically high.

Abbreviation: AUC, area under the curve.

Multicollinearity refers to the high correlated relation between variables; if variables are introduced in a model without having added value to each other, that can result in overcompensation and redundancy. This a particular concern for radiomics approaches, as this is better mitigated in deep learning. It is important to be aware of multicollinearity during the modeling process, as it can easily occur due to the competition of similar variables, and variables are wrongfully selected or omitted.

Model result interpretability can aid in the understanding of the modeling process, generalizability, and acceptance of a model by the medical community. The benefit of deep learning is that it allows for learning directly of the image itself and comes with the trade-off that it is much more a black box compared with radiomics machine-learning approaches, in which selected features, generally, can be compared with semantic understandings (e.g., tumor size and high heterogeneity). Efforts are arising to improve the human interpretability of deep-learning networks by visualizing the determining convolutional filters.^78,79 However, there is an unmet need for a transition from representation to derive the underpinning biologic processes or conditions detected by machine learning. For example, radiomics features derived algorithmically have been demonstrated to serve as representations of physiologic tissue-level characteristics, such as fattiness¹⁵ and physiologic activity⁸⁰ of the parotid glands associated with the development of xerostomia.

CHALLENGES IN ONCOLOGIC IMAGING USING RADIOMICS AND ARTIFICIAL INTELLIGENCE

The largest challenge for image-based machine-learning approaches is in consistent and large-scale curated and standardized image data acquisition. Ger et al⁸¹ showed that even if the same scanning protocol was used for CT image acquisition, this still resulted in differences in radiomics feature calculation on 100 scanners in 35 institutes. This illustrates that even CT, which is a much easier calibrated modality than PET or MRI, still suffers from substantial interand intrascanner variations. Many other studies have also illustrated the impact of different image reconstruction kernels, tube current, slice thickness, and scanner manufacturers on radiomics features.^82–84 This emphasizes both the importance of image acquisition standardization and calibration as well as testing the robustness of prediction image-based models in a new patient population that are scanned with different scanners or imaging acquisition protocols. This challenges interinstitutional implementation of prediction models.

For MRI, an additional hurdle is the lack of consistently available scans, which consequently results in small cohort sizes and lack of external validation in research studies. Systematic MRI acquisition is hampered due to continuous MRI sequence development, resulting in periodical changes in the image acquisition parameters and large differences in acquisition settings between treatment sites. Furthermore, the vast majority of the MR radiomics studies do not incorporate any intensity standardization of MR scans, which is questionable due to known biases in MR intensities.⁸⁵ A major, and often underestimated, challenge of MR-based AI applications is the lack of intrinsic value of MR intensities for the majority of acquisition methods (e.g., T1-/T2-/proton density–weighted images), meaning that they do not represent a specific fixed entity (i.e., they have arbitrary units), but rather have relative contrast value to adjacent MR intensity and structures (Fig. 6). MR normalization is currently still somewhat uncharted terrain outside the brain area.^86–88 Quantitative imaging modalities have been introduced, designed to represent diagnostically meaningful intensity values. Examples are diffusion-weighted imaging, T1-weighted or T2-weighted maps, and dynamic contrast-enhanced MRI,^89–91 yet currently, these MR sequences often have a limited signal-to-noise and courser resolution.

FIGURE 6. Pretreatment T1-Weighted MRIs Displayed With the Same Window Level (50–350 Arbitrary MR Units).

Clear differences can be observed in brightness (i.e., signal intensity) between the patients in tissues that are comparable in nature, such as fat and muscle.

REPORTING AND SHARING RADIOMICS AND ARTIFICIAL INTELLIGENCE APPROACHES

One major limitation is that processes and approaches of AI development and testing do not conform to historical frequentist hypothesis framework in many instances, thus making reporting of radiomics⁹² and AI models^93,94 difficult and representing a considerable barrier to advanced application for clinical use and external validation. Nevertheless, the field is evolving to more consistency and more rigorous modeling approaches. Many standardized guidelines are being introduced for modeling reporting (e.g., Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis [TRIPOD] methods),⁹⁵ image acquisition (e.g., Quantitative Imaging Biomarkers Alliance),⁹⁶ consistent image biomarker/ radiomics extraction (e.g., Image Biomarker Standardization Initiative),⁹⁷ AI-driven clinical trials⁹⁸ and general AI reporting,⁹⁹ and imaging-specific applications.¹⁰⁰ Moreover, to propel the field forward, more efforts are surfacing to publish data to make head and neck cancer imaging data FAIR (“Findable, Accessible, Interoperable, Reusable”)¹⁰¹ and thus spur new model building and validation. Similarly, through platforms like GIThub software/script publication, sharing and validation has become highly efficient. Finally, ontologies¹⁰² has made a concerted effort through improving the data annotation and informatics standards across data sources,¹⁰³ allowing facilitation of model development¹⁰³ and data transfer. These processes contribute to the implementation of robust actionable models in the head and neck cancer clinical practice and thereby consequently make it possible to tailor the treatment of head and neck cancer to improve therapy outcome and quality of life, allowing the transition of radiomics and AI from the laboratory to the clinic.

PRACTICAL APPLICATIONS.

• Radiomics and imaging artificial intelligence are rapidly being deployed in research-level applications in head and neck cancer.

• Personalized medicine using radiomics and artificial intelligence is necessarily dependent on the quality, robustness, and generalizability of generated prediction/classification models.

• Though exceptionally promising, rigor in model development, reporting, and implementation will require oncologists to have a general understanding of these approaches, their advantages over traditional methods, and their limitations.