Current State of Artificial Intelligence in Clinical Applications for Head and Neck MR Imaging

Abstract

Due primarily to the excellent soft tissue contrast depictions provided by MRI, the widespread application of head and neck MRI in clinical practice serves to assess various diseases. Artificial intelligence (AI)-based methodologies, particularly deep learning analyses using convolutional neural networks, have recently gained global recognition and have been extensively investigated in clinical research for their applicability across a range of categories within medical imaging, including head and neck MRI. Analytical approaches using AI have shown potential for addressing the clinical limitations associated with head and neck MRI. In this review, we focus primarily on the technical advancements in deep-learning-based methodologies and their clinical utility within the field of head and neck MRI, encompassing aspects such as image acquisition and reconstruction, lesion segmentation, disease classification and diagnosis, and prognostic prediction for patients presenting with head and neck diseases. We then discuss the limitations of current deep-learning-based approaches and offer insights regarding future challenges in this field.

Introduction

MRI is one of the important modalities for assessing patients with head and neck diseases in routine clinical practice. The region of the head and neck is characterized by a highly complex anatomy comprised of numerous organs within a narrow space, including the skull base, temporal bone, orbit, laryngopharynx, oral cavity, nasal/sinonasal cavity, major salivary glands, and many other organs/spaces. A major advantage of MRI in assessing head and neck diseases is its higher soft-tissue contrast in representative imaging sequences (e.g., T1-weighted imaging [T1WI] and T2-weighted imaging [T2WI]) compared to CT, thus making MRI highly useful to assess diseases in the head and neck region without radiation exposure.¹ In addition, several advanced MR techniques can provide functional information (e.g., diffusion-weighted imaging [DWI] and perfusion/permeability weighted imaging); these additional techniques provide useful information for assessments of head and neck diseases.^2–4 However, MRI of the head and neck still presents challenges in various clinical applications: (1) In image acquisition, due to the complex anatomical structure of the head and neck, images with higher spatial resolution are often required, leading to longer acquisition times, which can be problematic. The complex morphology of the head and neck can also result in B0 and B1 field inhomogeneity, which can make it difficult to obtain sufficient image quality. The presence of air in the nasal/sinonasal cavity or laryngopharynx and metallic substances in the oral cavity can cause magnetic susceptibility artifacts, which are also an important problem in image quality assessment.^2,5 (2) In lesion detection and segmentation, it can be difficult to detect lesions and to accurately segment lesions because of the anatomical complexity of the head and neck. Segmenting a lesion and normal anatomical structures around the lesion is crucial for radiotherapy treatment planning, but performing this task manually is very time-consuming.⁶ (3) In lesion diagnosis and differentiation, the signal intensity in the lesion is often nonspecific (e.g., low signal intensity on T1WI and high signal intensity on T2WI), which can be a source of difficulty for radiologists. Related to the disease diagnosis, it is also difficult to predict prognoses (particularly in cancer patients) because various factors such as lesion size, surface irregularity, intralesional heterogeneity, and several other factors affect the treatment outcome. Therefore, although MRI is indispensable for assessing head and neck lesions, these limitations should be resolved.

Over the past decade, advanced analytical methods such as texture analyses and radiomics have gained attention in the research field of radiology.⁷ Compared to the interpretation of conventional imaging findings, these advanced methods are capable of more accurately quantifying the information contained within the images at a higher level of detail, thus providing high diagnostic performance.^8–28 More recently, remarkable advances have been made in the development and clinical application of artificial intelligence (AI) in the field of radiology.^29–32 In particular, the analysis of deep learning using convolutional neural networks (CNNs) has become mainstream, and extensive clinical research on its application continues for various categories in medical imaging, including that of the head and neck (Fig. 1).^33–51 First, there has been remarkable progress in the development of deep-learning reconstruction techniques in image acquisition, focusing on both the reduction of image noise and reconstructions to high-resolution imaging (i.e., super-resolution); these techniques provide high-quality images and/or short scanning time-derived images. Second, in the segmentation of targets, the application of semantic segmentation with deep-learning algorithms has progressed, with reports of high-precision delineations of ROIs in various targets, including lesions such as malignant tumors and normal organs. Third, in the diagnosis or the differentiation of lesions, classifications using a deep-learning method have achieved very high diagnostic accuracy rates in various disease differentiations, suggesting that deep-learning methods may achieve diagnostic ability that is comparable to that of board-certified radiologists and to become useful as diagnostic support tools for all physicians. Fourth, as another classification task using a deep-learning approach, prognosis prediction (especially in patients with cancer) will be beneficial for the appropriate patient care. In the investigations that focused on the head and neck, the number of segmentation-related studies has been the highest compared to other categories, followed by disease classification, prognosis prediction, and image acquisition in recent studies using a deep-learning analysis (Fig. 2). In addition, although the utility of deep-learning analyses in the head and neck has been investigated mainly using CT, the number of reports investigating deep learning with other imaging modalities, including MRI, have been increasing (along with those of CT) (Fig. 3).

Fig. 1.

Representative artificial intelligence (AI)-based clinical appreciation in head and neck MRI. Representative clinical applications using AI-based methods are shown: image reconstruction with denoising, lesion segmentation, disease classification, and prognosis prediction from an imaging dataset. AI, artificial intelligence.

Fig. 2.

The numbers of head and neck imaging studies in various clinical applications using deep-learning approach in recent years. The search results in PubMed (https://pubmed.ncbi.nlm.nih.gov/) with ‘deep learning neck imaging (acquisition or reconstruction)’ were categorized as acquisition/reconstruction, ‘deep learning neck imaging segmentation’ as segmentation, ‘deep learning neck imaging classification’ as classification, and ‘deep learning neck imaging prognosis prediction’ as prognosis prediction. The data from 2020 to 2022 are presented.

Fig. 3.

The numbers of head and neck imaging studies in various modalities by a deep-learning approach in recent years. The search results in PubMed (https://pubmed.ncbi.nlm.nih.gov/) with ‘deep learning neck imaging CT’, ‘deep learning neck imaging MRI’, and ‘deep learning neck imaging PET’ were, respectively, categorized as CT-, MRI-, and PET-related investigations using deep learning. The data from 2020 to 2022 are shown. PET, positron emission tomography.

As mentioned above, there has been an increasing number of studies of technological innovations utilizing AI in the field of head and neck MRI, particularly with a focus on deep learning, and the studies have established the utility of AI in various applications such as image acquisition, lesion segmentation, disease diagnosis, and prognosis prediction. In this review, we comprehensively describe the usefulness of AI, especially deep learning, in the field of head and neck MRI, and we provide an overview of the demonstrated benefits and limitations as well as future perspectives.

Image Acquisition and Reconstruction

Numerous important anatomical structures are present within the small FOV of the head and neck, and high-spatial-resolution imaging is thus generally required for the sufficient quality of image reading. Scanning with a short acquisition time is also required because imaging of the head and neck region always suffers from the patient’s motion by, for example, respiration and swallowing. Susceptibility artifacts in the region located around the air space of the laryngopharynx and metallic items in the oral cavity are also problematic in the efforts to maintain good image quality. To address these problems, the designs of MRI sequences and acquisition methods have evolved and have been applied to the head and neck over the last decade or so. A short scanning time without a reduction of the SNR is considered a crucial task for head and neck MRI acquisition.

The methods for image acceleration that are most frequently used in daily clinical practice include parallel imaging, half-scan (half Fourier), and the zero filling interpolation, among others. A compressed sensing technique was introduced in 2015 in the main field of fast MRI acquisition.⁵² This technique is an image reconstruction method that uses the undersampling of k-space data with sparse image signals and a repeated denoising cycle performed by wavelet transformation. There are several studies in which a compressed sensing technique was applied in head and neck MRI.^53–55 Takumi et al. performed a comparative study of the quality of contrast-enhanced (CE) fat-suppressed 3D gradient-echo imaging with both quantitative and qualitative assessments between images with and without compressed sensing, using a dataset of patients with pharyngolaryngeal squamous cell carcinoma (SCC).⁵³ They reported that the images obtained with compressed sensing demonstrated better image quality compared to those without, and they concluded that compressed sensing improves the image quality of 3D-based CE-T1WI for the evaluation of pharyngolaryngeal SCC, without requiring additional acquisition time. Kami et al. also observed improved image quality of compressed sensing-based CE-3D-T1WI compared to the 2D multi-slice spin echo sequence in patients with maxillofacial lesions.⁵⁴ The superiority of a T2WI sequence with compressed sensing compared to the conventional parallel imaging-based sequence for the evaluation of normal structures in the oral cavity was demonstrated by Tomita et al.⁵⁵

More recently, deep-learning-based reconstruction algorithms have been reported worldwide.^56,57 In most of the assessments of the image quality of deep-learning-based reconstruction, the novel reconstruction method using a deep-learning-based reconstruction algorithm was observed to be the best method to obtain superb image quality in various image-reconstruction techniques; even the above-mentioned compressed-sensing technique tended to provide inferior image quality compared to the deep-learning-based reconstruction. The number of investigations that performed deep-learning-based MRI reconstruction for the evaluation of head and neck disorders is very limited. Naganawa et al. evaluated the contrast-to-noise ratio (CNR) of endolymph to perilymph in MRI examinations of endolymphatic hydrops based on 3D fluid-attenuated inversion recovery (FLAIR) imaging performed 4 hours after the intravenous administration of a contrast agent.⁴⁰ They assessed the hybrid of (i) the reversed image of the positive endolymph signal and (ii) the native image of the perilymph signal multiplied with the heavily T2-weighted MR cisternography images (named ‘HYDROPS-Mi2’) with a deep-learning-based denoising algorithm. Their results revealed that the CNR of the images obtained with deep-learning-based denoising (7738.6 ± 5149.2) was increased by more than fourfold compared to the images obtained without denoising (1681.8 ± 845.2).⁴⁰ In another study, Naganawa et al. performed the fast scanning of endolymphatic hydrops imaging with the use of HYDROPS2-Mi2 and a deep-learning-based denoising algorithm. They obtained sufficient image quality within a 5-minute imaging protocol with the help of deep-learning reconstruction, whereas the conventional imaging method with HYDROPS2-Mi2 required 12 minutes for image acquisition.⁴⁷

Deep-learning-based super-resolution is another featured technology in the field of deep-learning reconstruction. Koktzoglou et al. applied the deep-learning-based super-resolution technique to head and neck MR angiography (MRA).⁵⁸ This super-resolution model was created with the training for the estimation of difference between the high- and low-resolution images based on image domain deep learning method. They reported that deep-learning-based super-resolution (up to fourfold) as well as twofold lower-resolution input volumes provided the same quality of images obtained from high-resolution ground-truth volumes with a long scanning time, based on their evaluation with the Dice similarity coefficient (DSC), structural similarity (SSIM), arterial diameters, and arterial sharpness. They concluded that head and neck MRA with deep-learning-based super-resolution has the potential for an up to fourfold reduction in acquisition time.⁵⁸ Some of the structures that comprise the head and neck are especially small and anatomically complex. It has been difficult to achieve the sufficient depiction of several small normal structures (e.g., the inner ear, the parathyroid) and lesions (e.g., T1-stage cancers) by routine imaging sequences because of their smallness and/or the similarity of the signal intensity located near the target.⁵⁹ Super-resolution techniques could thus contribute significantly to the acquisition of good-quality and high-spatial-resolution images without increasing the acquisition time.

Artifact reduction is an important issue in the field of MRI. Head and neck DWI is challenging and often results in poor image quality with severe image distortion and susceptibility artifact due to the complex anatomical shapes and the presence of air and/or metallic substances. A study published in 2023 investigated the new technique of susceptibility artifact correction in an echo planar imaging sequence using an unsupervised deep neural model.⁶⁰ This technique can be effective for improving the image quality of the head and neck DWI.

Synthetic imaging between modalities (i.e., image-to-image translation) is another notable deep-learning-based reconstruction strategy. More specifically, bone MR imaging to synthetic CT techniques enables the visualization of cortical bone without the need for radiation exposure, which is particularly useful for pediatric patients. Although acquisition-based bone MRI technique was previously investigated, the zero TE imaging is one of the representative methods of bone MRI to visualize short-T2 materials (i.e., cortical bone);⁶¹ deep-learning-based bone MRI is also considered as an interesting technique for the bone MRI. Bambach et al. demonstrated the optimization that can be used to create a robust deep-learning model for the conversion of bone MR imaging to synthetic CT with Light U-Net architecture, and they suggested the possibility of using this technique in clinical practice.⁵⁰ Including the abovementioned deep-learning-based super-resolution technique and the synthetic imaging, the potential utility of these techniques is highly expected; however, derived images are created by incorporating data that were not originally acquired from the target and may contain artificially reconstructed false information. This is considered one of the potential important limitations of these types of deep-learning-based image processing. Such images should be carefully reviewed under the close supervision of a radiologist.

Segmentation

Target delineation and segmentation is important to accurately understand the main localization of lesions and the range of disease. Several research groups have suggested the utility of lesion delineation or detection of the lesion extent and segmentation using several modalities such as CE-CT, non-CE- or CE-MRI, and ¹⁸F-fluorodeoxyglucose positron-emission tomography/CT (FDG-PET/CT).^62,63 The accurate and rapid accomplishment of this procedure clinically is thus desired. More specifically, target segmentation is crucial in head and neck imaging, especially for successful radiotherapy procedures. For treatment planning in radiotherapy, accurate delineation of the target lesion and the organs at risk is mandatory, and the delineation procedure directly influences the quality of the radiotherapy.^64,65 Because various types of soft tissue are present in the head and neck, MRI is an effective modality due to its excellent contrast of soft tissue compared to other modalities (e.g., CT) for delineating the target lesion and organs at risk, and it has thus become a more common imaging technique for the delineation in radiotherapy treatment planning.⁶⁶ However, the delineation of target lesions and organs at risk in the head and neck region is a very labor-intensive, time-consuming, and observer-dependent procedure. The delineation process is typically performed by radiologists or radiation oncologists, and it takes an average of ~3 hours to delineate a full set of tumor volumes and organs at risk, mainly because of the complex anatomy of the head and neck.⁶ To address this limitation, AI-based segmentation methods, particularly deep-learning-based techniques, have been widely introduced as novel supportive tools and are investigated with excellent initial results, as follows.

Investigations of MR-based ROI delineation using deep-learning methods for head and neck cancers or organs at risk have been performed in patients with nasopharyngeal cancer^67–70 and oropharyngeal cancer.^71,72 To delineate the appropriate ROIs, most of these investigations used the deep-learning architecture of an Encoder-Decoder-type CNN, and the 2D U-net or 3D U-net was the most popular type of this model. Automatic MRI-based delineation for the primary lesion of nasopharyngeal cancer using CNNs has been applied in several studies.^38,67–70 For example, Lin et al. used U-net architecture for the delineation of the primary lesion of nasopharyngeal cancer on CE-T1WI.⁶⁷ They reported the DSC of 0.79 with the ground truth by an expert radiation oncologist or radiologist’s manual assessment, whereas the DSCs by qualified radiation oncologists ranged from 0.71 to 0.74. The use of their CNN model as an assistive tool for radiologists also reduced the intraobserver variation (the interquartile deviation of DSC) by 36.4%, and it reduced the contouring time by 39.4% (from 30.2 to 18.3 minutes).⁶⁷ Chen et al. proposed a novel multi-modality MRI fusion network (MMFNet) in which T1WI, T2WI, and CE-T1WI sequences were well integrated to provide a complete accurate segmentation of nasopharyngeal cancer. Their model consists of a multi-encoder-based network and one decoder to capture modality-specific features. Their study achieved the DSC of 0.72, which was higher than that obtained in a U-net-based analysis with a single sequence of T1WI, T2WI, or CE-T1WI.⁶⁸ Although Chen et al.’s study was performed with a multi-modal evaluation, the DSC value they reported is lower than that obtained in the above-cited study by Lin et al. This might be due to the variability in the two studies’ patient cohorts. In contrast, Wong et al. demonstrated the performance a U-Net-based CNN using only the non-CE-sequence (i.e., fat-suppressed T2WI), and their results showed no significant difference in the DSC (0.71) compared to the CE-T1WI sequence.³⁸

Investigations of the segmentation of oropharyngeal cancer have also been reported.^71,72 Rodríguez Outeiral et al. obtained the best DSC values with the combination of multiple MRI sequences (T1WI, T2WI, and CE-T1WI) as input using 3D U-Net. Notably, their DSC results suggested that manually reducing the context around the tumor, which they defined as a semi-automatic method, results in better segmentations than those given by a completely automatic method; with the semi-automatic method the DSC value was 0.74, whereas with the completely automatic method the DSC value was 0.55.⁷¹ Wahid et al. also demonstrated improved delineation of oropharyngeal cancer lesions with the combination use of conventional sequences (T1WI and T2WI) and quantitative functional image sequences; they used the apparent diffusion coefficient (ADC) from DWI and the volume transfer coefficient (K^trans) and the volume of the extravascular extracellular space (Ve) from dynamic CE (DCE) perfusion as input channels to a 3D Residual U-Net.⁷²

The automatic delineation of organs at risk is also an important task in patients with head and neck cancer who need radiotherapy planning — especially the planning of intensity-modulated radiation therapy (IMRT). Dai et al. used the datasets of both CT for bony-structure contrast and MRI for soft-tissue contrast in model training for the delineation of multiple organs at risk (totally 18 organs) with the CNN, resulting in high DSC values (approximately 0.8–0.9).⁷³ Interestingly, their study acquired the MRI data not directly; rather, it was acquired from the CT dataset by a pre-trained cycle-consistent generative adversarial network (GAN).⁷³ In contrast, Korte et al. investigated the delineations of the parotid gland, submandibular gland, and neck lymph nodes by using a T2WI dataset with multiple 3D-U-Net systems, resulting in DSC values of approximately 0.8.⁷⁴

The segmentation of other lesions and anatomical structures such as vestibular schwannoma in the cerebellopontine angle,^75,76 the inner ear and its related structures (e.g., cochlea, vestibule),^77–79 and venous malformations of the neck⁸⁰ has been described. Segmentation of the inner ear and its related structures necessary for the diagnosis of endolymphatic hydrops has also been a concern (see the section below titled ‘Disease classification and diagnosis’) and would be valuable for clinical practice.

Due to the excellent soft-tissue contrast obtained from MRI, MRI-based segmentation could contribute to the highly accurate segmentation of various lesions and organs at risk. The currently used segmentation process is usually performed for treatment-planning CT, but the combination use of multiple modalities, including MRI, is expected to be established in the future as a more useful method. Unfortunately, most of the studies mentioned above did not use external validation; they used internal validation with a hold-out or cross-validation scheme for the test set when assessing the created model’s accuracy. The performance of each new deep-learning model must therefore be interpreted carefully before its use in clinical practice.

Disease Classification and Diagnosis

There is a wide variety of differential diagnoses in head and neck imaging depending on the various anatomical origins. The ability to accurately determine the final diagnosis with high diagnostic confidence using images can help reduce unnecessary invasive examinations and surgical procedures. Numerous radiological investigations have described lesion enhancement patterns, lesion morphological shapes, and morphological changes in lesions over a given follow-up period, and several additional parameters obtained from functional imaging techniques are valuable for the diagnostic prediction of the histological type or differentiation of benign and malignant lesions in various head and neck diseases.^81–83 However, there are still many nonspecific imaging findings in head and neck MRI, and standard imaging sequences such as T1WI and T2WI can be limited in their ability to reliably differentiate, for example, benign from malignant lesions, resulting in difficulty reaching a diagnosis even among experienced radiologists. Advanced analytical methods such as a texture analysis or radiomics approach have been reported to provide mostly equivalent diagnostic accuracy (or sometimes better accuracy) in classifying head and neck diseases compared to classifications by expert radiologists who specialize in head and neck imaging.⁸⁴ There is a great need for a supplementary tool, and AI-based methods have the potential to make a significant contribution to this application. We next describe the aspects of the usefulness of AI-based techniques, especially deep-learning methods, for the diagnoses and classification of head and neck diseases.

Major salivary gland tumors; parotid tumors

The origin of approximately 80% of salivary gland tumors is a parotid gland. Pleomorphic adenoma is the most common histological subtype of parotid tumors, followed by Warthin tumor. Mucoepidermoid carcinoma is the most common subtype in malignant tumors. MRI is often used to investigate a parotid tumor’s diagnosis based on various imaging findings (mainly T2WI and CE-T1WI).⁸⁵ However, the imaging findings of parotid gland tumors are sometimes unspecific and often difficult to diagnose even by radiologists who specialize in the head and neck. Chang et al. demonstrated the differentiation of parotid tumors among pleomorphic adenoma, Warthin tumor, and malignant tumors with the use of 2D U-Net and five types of MRI sequences (T1WI, T2WI, CE-T1WI, DWI with b-value of 0 s/mm² [DWIb0], DW with b-value of 1000 s/mm² [DWIb1000], and ADC) with a fully automated method. They achieved high overall diagnostic performance with a DWI-based model consisting of DWIb0, DWIb1000, and ADC (0.71–0.81 accuracy).⁸⁶ Interestingly, combining DWI and other sequences did not improve the prediction accuracy. Matsuo et al. performed the integration of T1WI and T2WI into single pseudo-color images and then conducted an analysis with the combination use of a VGG16-based deep learning-model and the L2-constrained softmax loss function. The highest area under the curve (AUC) value to differentiate between benign and malignant parotid gland tumors (0.86) was obtained with this method, higher than that obtained visually by a board-certified radiologist (0.74).⁸⁷ More recently, the ‘transformer’ method has been applied to computer vision, including disease classifications in medical imaging; this technique uses the self-attention function which enables early information collection and combining of the total global information by dividing images into several parts.⁸⁸ Dai et al. used an MRI dataset and the combination use of a CNN and the transformer method for the classification of parotid gland tumors.⁸⁹ In their created model, named as TransMed, the CNN was used to extract a low-level feature from multi-modal imaging dataset; the output of CNN was conducted by a linear projection layer to create a low-level feature-based map. This low-level feature map was analyzed by the transformer layer to provide the final classification. Their proposed model achieved the diagnostic accuracy of 0.89 in differentiating the parotid gland tumors, which was superior compared to a CNN model’s performance.⁸⁹ Liu et al. performed a two-step approach using Res-Net deep-learning architecture and then the Transformer network. This two-step method employed aligns with the technique utilized in the aforementioned study by Dai, et al. First, a vector describing image features was produced using image dataset through the Resnet-18. Subsequently, a sequence classification network by transformer model identified the subtype of tumors using the extracted image by Resnet-18 as an input. Their results demonstrated that the multiple sequences-based model with T2WI, CE-T1WI, and DWI produced the best results with 0.85 accuracy to differentiate between malignant and benign parotid tumors.⁹⁰

Head and neck squamous cell carcinoma

Head and neck SCC is the most common malignancy arising from various regions of the head and neck. The main anatomy includes the pharyngolarynx and the nasal/sinonasal cavity. The correct diagnosis of this pathology is crucial for prompt decision-making regarding treatment planning in appropriate patient care. Deep-learning-based approaches have been assessed as a potential supportive tool for the rapid and precise diagnosis of these head and neck cancers. Wong et al. performed a discrimination analysis between early-stage (i.e., T1 stage) nasopharyngeal cancer and benign hyperplasia, using fat-suppressed T2WI with the Residual Attention Network and 3D volume input and multiple 2D acquired slices.⁹¹ They achieved high diagnostic accuracy (0.92) with a CNN-based algorithm, and this value was not significantly different from the diagnostic performance of an experienced radiologist. In another study, deep-learning-based T-staging in patients with nasopharyngeal cancer was investigated using imaging datasets of T1WI, T2WI, and CE-T1WI with the deep-learning architecture of ResNet.⁹² Those authors used the ‘weakly supervised learning’ method in which slice-by-slice analyses of multiple slices are conducted, after which the slice with the highest score (named the T-score) was used in the model’s training procedure. Their created model achieved 0.76 accuracy in the determination of T-staging. In patients with nasal or sinonasal cavity tumors, the differentiation between benign inverted papillomas and those with malignant transformation to SCC is important but challenging, particularly by imaging findings alone. Liu et al. investigated an MRI-based deep-learning approach to accomplish this challenge.⁹³ They performed a 3D-CNN analysis using T1WI with or without a contrast agent and/or T2WI, resulting in the highest diagnostic accuracy (0.78) with their developed CNN model, named ‘All-Net,’ a 3D CNN model with four convolutional layers followed by rectified linear unit activation functions and max pooling layers.

Lesions in the temporal bone

Diseases in the temporal bone for which an MRI-based diagnosis is useful include cholesteatoma, Meniere’s disease (i.e., endolymphatic hydrops), vestibular and facial schwannoma, and many others. In particular, the diagnosis of endolymphatic hydrops in medical imaging can be attained exclusively through the utilization of MRI incorporating specific techniques. In the past, an intratympanic injection of a gadolinium-based contrast agent was required to obtain an SNR that was sufficient to visualize the endolymphatic hydrops. However, with the widespread use of high-field MRI and the optimization of imaging protocols, endolymphatic hydrops can be well detected with an intravenous (not intratympanic) injection of a contrast agent and image acquisition 4 hours later, and this imaging sequence has been well developed for clinical use.^94–97 A deep-learning-based supportive tool for the assessment of endolymphatic hydrops was reported in 2020. Cho et al. described the combination use of automatically segmented images of the cochlea and vestibule by a deep-learning method with VGG19-based architecture and post-CE-3D-FLAIR images to obtain diagnostic information that indicates the degree of endolymphatic hydrops. This deep-learning-based output was highly consistent with the results obtained manually by experts, with an intraclass correlation coefficient of 0.971.⁷⁷ Park et al. further improved this deep-learning-based model by developing it with the use of the architectures of the Inception-v3 and U-Net combination use; the newly developed system successfully achieved the automatic selection of representative images from a full MRI dataset with a very short total analysis time: 3.5 seconds.⁷⁹

In disease classification and diagnoses using image datasets, deep-learning techniques are considered a promising method and have provided high overall diagnostic performance. However, most of the above-cited studies did not use images with the whole FOV for preparing the input used to train the deep-learning model. Rather, they used segmented images that include the lesion and exclude the area outside the lesion obtained by a manual procedure. A fully automated approach would be in great demand for easier clinical use because approaches using manual methods require high amounts of time and human effort. This methodological background may impede the worldwide use of deep-learning-based applications in clinical practice. Fully automated procedures with lesion detection, delineation, and classification abilities are desired for practical use in the field of diagnostic imaging. As in the studies of segmentation, most of the investigations of disease classification and diagnoses used internal (and not external) validation for the test set in order to assess the model’s performance. The diagnostic performance described in these reports might thus be higher than the real-world accuracy.⁹⁸ This limitation should be addressed by future studies.

Prognosis Prediction

Challenges in predicting the prognoses of patients with head and neck lesions, for example, predictions of the initial treatment outcome and long-term predictions of the disease control, are still difficult. Prognosis prediction contributes to the optimization of treatment strategies, including the selection of initial therapy and the indication for adjuvant therapies, and the prediction is also considered useful to validate the treatment plan for each patient with head or neck cancer. Over the past few decades, imaging methods have been well developed and used together with advanced imaging techniques such as DWI and DCE,^99–110 or a high-dimensional analytical method such as a texture analysis or radiomics method^{14,22,24,111,112} in various diseases as well as head and neck lesions. However, achieving accurate predictions is still challenging. Recently developed deep-learning-based techniques can make significant contributions to the prediction of patient prognoses as effective support tools with higher accuracy compared to MR imaging biomarkers and older analytical techniques.

Among the prognosis prediction studies using deep-learning-based methods with an MRI dataset, the number of studies of nasopharyngeal cancer is relatively high, and each of these studies assessed the prognosis in patients receiving chemoradiotherapy. As an initial result, Qiang et al. reported the utility of deep-learning-based MRI scores using T1WI, T2WI, and CE-T1WI.¹¹³ Their analysis was performed using MR images fed into 3D-DenseNet, and the low-dimensional features were then analyzed using DeepSurvivalNet and outputted as an MRI score. A combination diagnostic model of the clinical features and the deep-learning-based MRI score by a conventional machine learning algorithm of XGBoost were finally created. The model described by Qiang et al. demonstrated a statistically significant improvement in prognosis prediction with a Concordance Index (C-index) around 0.7–0.8 compared to the conventional tumor, node, metastasis (TNM) staging system. Notably, they used whole FOV images rather than manually segmented images, and they used external validation for the test set with the C-index of 0.72–0.76; the results of this analysis might be valuable toward the use of deep-learning-based prognosis prediction in other institutions.¹¹³ Zhong et al. investigated a deep-learning-radiomics model to predict the prognoses of patients with T3-stage nasopharyngeal cancer.¹¹⁴ They performed deep-learning-based feature map extraction by SE Res-Net using manually segmented T1WI, T2WI, and CE-T1WI. These deep-learning-based features were finally integrated by radiomics signature building. The created model showed excellent prognostic ability for disease-free survival, with the C-index of 0.8–0.9.¹¹⁴ Li et al. reported the utility of an ensemble learning method that effectively combined two sets of single deep-learning-based results for the prediction of the prognosis of patients with nasopharyngeal carcinoma at an advanced clinical stage.¹¹⁵ They used the Resnet-V2 deep-learning architecture with pre-treatment and post-treatment MRI, finally combining the datasets into one deep-learning-based diagnostic model. The ensemble learning-based diagnostic model exhibited the best performance compared to a single deep-learning-based model and conventional clinical staging, probably because the combined use of two datasets effectively improved the model’s performance. Li et al. also performed an analysis of gradient-weighted class activation mapping (Grad-CAM) images.¹¹⁵ Grad-CAM is used for visualization when a deep-learning diagnostic model is used to look at a target image. Interestingly, the deep-learning model suggested that the areas around the tumor and some cervical lymph nodes were strongly related to the prognosis of the tumor, whereas the relationship between the signal of the primary tumor area and the prognosis was not as strong as expected in many cases.¹¹⁵ In another study, Li et al. investigated deep-learning-based prognosis predictions and conducted a Grad-CAM analysis using several sizes of FOVs fed into the deep-learning model, and they observed a similar tendency: the area around the tumor is the most important for predicting the prognosis of patients with nasopharyngeal carcinoma.¹¹⁶

Regarding other primary sites of head and neck cancers, Tomita et al. investigated the utility of DWI with a deep-learning analysis in patients with hypopharyngeal or laryngeal SCC treated by definitive chemoradiotherapy. They trained the deep learning of Xception architecture by using anatomically masked DWI including the primary lesion at two time points (pre-treatment and intra-treatment). The results revealed that a deep-learning model trained with intra-treatment DWI provided the highest diagnostic performance to predict local recurrence, with the accuracy value of 0.767.¹¹⁷

In addition to directly estimating the prognosis, it is important to predict the genetic information that is well known as an independent prognostic factor, such as human papillomavirus (HPV) in oropharyngeal cancer and Epstein-Barr virus (EBV) in nasopharyngeal cancer. Image-based parameter integration and calculation for the prediction of the genomic information is recognized as ‘radiogenomics.’¹¹⁸ Research using deep-learning-based radiogenomics is still in progress, but the use of radiogenomics in clinical practice may be realized in the near future.

Duan et al. investigated the MRI findings of patients with large vestibular aqueduct syndrome and performed a deep-learning analysis to predict the patients’ hearing prognosis. They used images from a segmented T2-weighted sequence to include the target anatomical structure for the evaluation. The GoogLeNet-trained diagnostic model provided the best diagnostic accuracy (0.98) to differentiate between patients with stable versus fluctuating hearing loss.¹¹⁹

Prognosis prediction is essential for the determination of the best treatment planning and the appropriate post-treatment strategy, particularly in patients with cancer of the head and neck.¹²⁰ Few studies have investigated this, but more precise patient classification might be feasible in the near future. Because MRI provides excellent contrast of various tissues and a large amount of information about the target lesion, combinations of a deep-learning method and the information derived from MRI will bring the diagnostic performance to a higher stage.

Future Perspective

Analyses of deep-learning techniques using MRI datasets are expected to be useful for a variety of purposes, including image acquisition/reconstruction, target segmentation, disease classification/diagnosis, and prognosis prediction. Such innovative methods will bring about a higher level of diagnostic performance compared to conventional methods. However, several issues remain. First, almost all of the studies related to head and neck MRI described above were performed with a transfer learning technique, whereas the CNN architecture used in the studies were quite different. In addition, some of the studies used a single CNN network, but other studies combined multiple architectures into one diagnostic model. Moreover, a wide range of methods for setting hyperparameters was present. The majority of the investigations cited above applied various deep-learning architectures and hyperparameters. With this background, it is challenging to establish a universally applicable model setting method, and it might be necessary to optimize the technical issues in model development based on each institution. A recent report by Sellergren et al. described a solution to address this limitation. Their general pretrained model followed by additional pretraining by medical images achieved higher model accuracy, and this technique may therefore contribute to the generalization of the pretraining procedure.¹²¹ It might also be helpful to build a basic model using large-scale medical images before the individual use; it could be effective to first build a fundamental deep-learning diagnostic model using a large sample size as the basis to accommodate image-quality differences due to multiple vendors and varying parameter settings, and then conduct fine-tuning of the model for individual uses for various purposes. Such an approach may help unify the processes of deep-learning techniques. In addition, the use of eXplainable AI (i.e., XAI) is an important method that can improve the utility of deep-learning algorithms for routine clinical practice. This function will aid the human interpretation of deep-learning-based diagnoses more objectively. A Grad-CAM approach as a type of XAI has been used frequently in recent investigations to identify the site at which a deep-learning model focuses (Fig. 4). Second, instead of using the entire images (those including the full FOV) as the input in model training, most of the prior analyses have been of the images after a manual segmentation of the lesion by a physician. Particularly, as described above in the ‘Disease classification and diagnosis’ section, most of the studies were performed using the segmented images for model training and testing. A full FOV image might be difficult to analyze for appropriate lesion detection/segmentation or diagnosis, probably because the complex anatomical structure of the head and neck includes a large amount of information. However, even with localized input data as well as a full FOV input, AI might output data by focusing on different parts of the target areas as the basis for decision-making.¹¹⁵ Such findings can also be revealed by using the above-mentioned function of Grad-CAM. Moreover, only a few studies have used external validation for the test set to estimate their diagnostic models. The deep-learning model created in such a situation might result in very limited use; the created model is likely to be used only in its developers’ institution. A review published in 2022 also indicated that among the AI-related studies focused on the head and neck, the number of investigations with the use of external validation in the test session is relatively small.¹²² For an appropriate external-validation procedure, the use of large amounts of data from public academic institutions (e.g., J-MID from the Japan Radiology Society) may be one of the steps to solve this problem.¹²³ Further research is necessary to address these limitations, as are discussions to integrate the existing knowledge and future aspects. From this perspective, we speculate that the incorporation of deep-learning-based clinical tools into daily clinical practice may require more time. However, we believe that with the further development and careful monitoring of appropriate approaches and basic/clinical investigations, clinically significant achievements can be made.

Fig. 4.

An example of gradient-weighted class activation mapping (Grad-CAM) for a deep-learning analysis targeting head and neck cancer. (a) Left maxillary cancer is observed on fat-suppressed T2WI (arrows). (b) The image shows the Grad-CAM displaying where the deep-learning model will be looking; this colormap represents a mathematical computation of the degree to which each pixel influences the final diagnosis; pixels with a stronger degree of influence are represented with a more intense red on the red-green-blue (RGB) color scale. In this case, the model analyzed the presence or absence of head and neck cancers on the MRI. Hot spots can be easily identified on the tumor lesion in this map (arrows). Grad-CAM, gradient-weighted class activation mapping; RGB, red-green-blue; T2WI, T2-weighted imaging.

Conclusion

AI-related techniques, particularly deep learning, are considered promising for the assessment of head and neck MRI, including image acquisition/reconstruction, segmentation, classification/diagnosis, and prognosis prediction. However, several limitations must be addressed before the deep-learning techniques can be applied in general medical practice in the field of radiology, and all of the future development of AI-based techniques should be closely monitored.