AI-PREDICT-BM: Artificial Intelligence to predict resectability and evaluate decisions for induction chemotherapy in treatment of buccal mucosa cancer – A novel pilot study

Abstract

Background:

Diagnosis and treatment of Carcinoma Buccal Mucosa is dependent on imaging techniques such as contrast-enhanced computed tomography (CECT), which is primarily used to stage the disease and predict resectability. Recent studies have identified a ‘Borderline Resectable’ subgroup in these patients who benefit with induction chemotherapy prior to surgery.

Materials and Methods:

This prospective observational pilot study, from April 2022 to March 2024, curated a dataset of 256 preoperative CECT scans of patients with stage IVA and IVB squamous cell carcinomas of the buccal mucosa, which were integrated into a novel artificial intelligence-based machine learning model designed to predict resectability for upfront surgery. We developed a Convolutional Neural Network-based predictive model to distinguish between “Borderline Resectable” and “Resectable Upfront” disease.

Results:

The model displayed high performance with an overall F1 score of 0.8, efficiently stratifying tumors based on resectability. Integration with Gradio allowed access to run the model on a local server, which allowed real-time execution of the model. The area under the curve (AUC) for the training set was 0.9652, with 50.39% sensitivity, 96.65% specificity, 65.75% negative predictive value, and 94.20% positive predictive value. The validation set had an AUC of 0.9735, along with 98.40% specificity, 67.96% negative predictive value, 55.73% sensitivity, and 97.33% positive predictive value.

Conclusion:

This study represents a first step toward the use of artificial intelligence to aid in the treatment to of patients with carcinoma buccal mucosa, allowing us to avoid the possibility of margin positive resection with upfront surgery.

INTRODUCTION

The burden of carcinoma of the buccal mucosa in the Indian subcontinent is a matter of great concern. It is generally related to behavioral habits of patients, which include tobacco consumption. Computed tomography (CT) is a vital diagnostic tool for head and neck malignancies, for accurate diagnosis and appropriate surgical planning. However, variations in CT reporting constitute a crucial gap. Stage IV tumors are described in the AJCC 8^th edition as T4a and T4b lesions, which were popularly classified among clinicians as operable and inoperable, respectively. Yet, this classification overlooks variations in resectability, particularly in T4b disease where upfront resection may result in postoperative margin positivity. CT scans are reported with various degrees of accuracy and often by nononcological radiologists who fail to categorize a resectable and a nonresectable disease. In order to standardize CT scan reports and surgical decision making for head and neck cancer, a novel Artificial Intelligence-driven Deep Learning model was developed in this study that uses CECT images to predict surgical resectability. The model can interpret a CECT image that has been uploaded from a computer and analyze it to determine if the disease is Resectable Upfront or Borderline Resectable. This allows for the identification of patients in the latter group who may benefit from induction chemotherapy prior to surgery.

MATERIAL AND METHODS

Study design

This was a prospective observational pilot study and was conducted from April 2022 to March 2024, after obtaining ethical clearance from the Institutional Review Board. The study population included patients with carcinoma of the buccal mucosa, presenting to our institution. These patients with operable disease were segregated into two groups: Resectable Upfront and Boderline Resectable. Unresectable patients or those who required nonsurgical treatment were excluded from this study. The inclusion and exclusion criteria for this study were as follows:

Inclusion criteria

Clinical or Biopsy-proven Stage IVA and IVB Squamous Cell Cancers of the Head and Neck arising from the Buccal Mucosal with preoperative CECT Scans of the Head and Neck depicting the following:

1. T4a disease causing bony destruction or involvement of the skin

2. Tumor infiltrating into the skin and buccal space with edema or small orocutaneous fistula up to the zygomatic arch or satellite nodules visible on CECT scans

3. T4b tumors involving the masticator space or the pterygoid plates or any muscle of mastication (temporalis, lateral and medical pterygoid or masseter)

4. High infratemporal fossa involvement in axial CECT scan slices where the tumor reaches above the sigmoid notch of the mandible (Supra-notch disease)

5. Ipsilateral lymph nodes without extra nodal extension.

Exclusion criteria

This includes ‘Grossly Unresectable’ or Metastatic disease or Carcinoma of the Oral Cavity with primary tumor in locations other than the Buccal Mucosa:

1. Carcinoma of the lip or commissures or bony involvement of the central arch of the mandible and the floor of the mouth

2. Stage IVC or metastatic disease

3. Bilateral lymphadenopathy with extranodal extension

4. Tumor or nodal burden in the neck encasing the internal carotid vessels or involving the skull base, prevertebral fascia, and pharynx.

Organization of the CECT scan images for analysis

The sample included 256 CECT scan images, each comprising four slices, which was curated as a dataset, which best depicted the resectability of the tumor. These included axial, coronal, and sagittal slices, which were then merged together into a single image collage comprising four images. These images were meticulously collated into a comprehensive dataset and categorized into two distinct groups: ‘Borderline Resectable,’ which included tumors reaching up to the high infra temporal fossa (supra-notch disease) and extensive skin involvement by the tumor or edema (hyperdense thickening on CT Scan) reaching up to the zygomatic process where upfront resection would result in margin positivity, thus indicating tumors potentially benefiting from induction chemotherapy prior to surgical intervention, and ‘Resectable Upfront,’ which included low infratemporal fossa (infra-notch disease), disease localized to the masticator space, and buccal space suggesting tumors suitable for direct surgical resection without preoperative systemic therapy. The dataset was structured into a separate training set and a validation set to ensure proper evaluation after the model was constructed.

Data collection and preparation

The dataset utilized in this machine learning model was sourced from the uploaded CT scan images, which were grouped into folders from our Google Drive and subsequently imported into Google Colaboratory, which utilizes a Juypter notebook to write programs using computer coding with the Python software. The training set included 60% (n = 154) of the images, which were grouped into Borderline Resectable (n = 77) and Resectable Upfront (n = 77). The validation set included 40% (n = 102) of the dataset, which was also grouped similarly as Borderline Resectable and Resectable Upfront groups (n = 51 in each group). To enhance the generalization ability of our model to account for variations in imaging conditions, we employed data augmentation techniques, including shear, zoom, and horizontal flip, on the training set. Additionally, standardization of image sizes to 256 × 256 pixels and scaling of pixel values to the range (0, 1) were performed to facilitate consistent model training.

Deep learning model architecture

A Convolutional Neural Network (CNN) model was utilized as our predictive framework due to its effectiveness in image classification tasks. The architecture of the CNN comprised three convolutional layers followed by max-pooling layers, designed to extract essential spatial features from input CECT scan images. Each convolutional layer utilized Rectified Linear Unit (ReLU) activation functions to introduce nonlinearity and enhance feature representation. Subsequently, max-pooling layers were employed to down sample feature maps, reducing computational complexity while preserving crucial information. The flattened output from the convolutional layers was then passed through two fully connected layers, each containing 128 virtual neurons, facilitating higher-level feature abstraction. Finally, a sigmoid activation function was applied to the output layer to produce binary classification results indicating tumor resectability status. This complex algorithm is summarized in the following figure [Figure 1].

Figure 1.

Outline of the Convoluted Neural Network machine learning model

Model training

Training of the model was conducted using the Adam optimizer with a learning rate set to 0.001, while the binary cross-entropy loss function was employed to optimize model performance. The training process spanned five epochs, with a batch size of 16, and validation data were incorporated to monitor the model’s performance and prevent overfitting. By iteratively adjusting model parameters based on feedback from the validation set, we aimed to optimize the model’s predictive capabilities and ensure robust generalization to unseen data.

Evaluation and statistical analysis

The trained CNN model’s performance was comprehensively evaluated using the test dataset to assess its ability to predict tumor Resectability accurately. Various performance metrics including the F1-score were computed to quantify the model’s classification performance. The obtained results were statistically analyzed to assess the significance of the model’s performance, the overall sensitivity, specificity, positive predictive values, and negative predictive values, which were calculated from an Receiver Operating Characteristic (ROC) curve that was calculated within the Google Colaboaratory notebook. The categorial variables were grouped as percentages, and continuous variables were represented as mean and standard deviation.

Integration with Gradio

In addition to model development and evaluation, we sought to enhance the accessibility and usability of our predictive framework by integrating it with Gradio, a user-friendly graphic interface platform, wherein a streamlined interface facilitates uploading of CECT scan images formatted similarly in a collage and obtain real-time resectability predictions, for demonstration purposes, which runs on a local server only. A screenshot of the same is demonstrated in the following figure [Figure 2].

Figure 2.

Interface of the Image classifier after development of the application on Gradio

RESULTS

Clinicoradiological description of the data sets

The dataset includes 154 patients in the training set and 102 patients in the validation set. The patient’s clinical and demographic data were comparable. Patients in both groups had an average age of 50 years. Both groups have a male predominance. Tumors were mostly distributed on the right side, with 54% and 66% in the training and validation sets, respectively. Node-positive disease was common, involving 85% and 92% of individuals in the training and validation sets, respectively. Notably, around 50% of individuals in each set were Resectable Upfront, while the remainder demonstrated infratemporal fossa involvement and were considered as Borderline Resectable. The following has been summarized in Table 1.

Table 1.

Sociodemographic and clinical characterization of the training and validation sets

	Training Set (n=154)	Validation Set (n=102)
Age	50±11.34	49±8.16
Gender (Male)	123 (79.8%)	88 (86.27%)
Side of Tumor (Right-Sided)	83 (53.89%)	67 (65.6%)
Node Positive Disease	130 (85%)	94 (92.15%)
Skin Involvement	67 (43.50%)	59 (57.48%)
Underlying Bone Involvement	119 (77.27%)	84 (82.35%)
Buccal Space Involvement	99 (64.28%)	49 (48.03%)
Masticator Space Involvement	73 (47.70%)	54 (52.94%)
Resectable Upfront	77 (50%)	51 (50%)
Boderline Resectable	77 (50%)	51 (50%)

Analysis of the model during training and validation

During the first epoch, the model had a relatively high training loss of 0.76 and a validation loss of 0.54. However, the accuracy was moderate for both training (67.97%) and validation (68.75%) datasets. As training progressed, both training and validation loss decreased significantly, indicating improvement in the model’s performance. By the fifth epoch, the model achieved very low training loss (0.01) and validation loss (0.0004), indicating good convergence. Additionally, both training and validation accuracy reached near-perfect values, indicating that the model was able to accurately classify the majority of the samples in both datasets [Figure 3a and b]. The model achieved an overall F1 score of 0.8, signifying its strong performance in accurately classifying the CT scan images.

Figure 3.

(a) Accuracy per Epoch. (b) Loss per Epoch

ROC curves analyzed the overall performance of the model. For the training set, the AUC was 0.9652, with a sensitivity of 50.39%, a specificity of 96.85%, a negative predictive value of 65.75%, and a positive predictive value of 94.20%. The validation set had an AUC of 0.9735, with a sensitivity of 55.73%, a specificity of 98.40%, a negative predictive value of 67.96%, and a positive predictive value of 97.33% [Figure 4].

Figure 4.

Comparison of the ROC curves of the training and the validation sets

DISCUSSION

Carcinoma of the buccal mucosa represents a major health concern in the Indian subcontinent and is primarily linked to the sociobehavioral attributes of tobacco consumption. On average, 85% of these patients present late due to neglect, and the most common stage of presentation is Stage IVA or IVB. Patients who have unresectable disease may survive up to 2–12 months without any surgical management.[1] Diagnosis of Head and Neck Cancer is dependent on Contrast-Enhanced Computed Tomography (CECT), which serves as the primary investigative tool for staging, surgical planning, and determining resectability of the disease. The American Joint Committee on Cancer (AJCC) 8^th Edition determines the staging of these cancers using a TNM Staging system, where the T stage determines the amount of surgical resection required for adequate clearance of the tumor. However, it describes the T4a stage as Resectable, and the T4b stage as Unresectable, which is a gray area when it comes to surgical planning. Since the resection involves a three-dimensional assessment of margins, these tumors may have sinister outcomes when they involve multiple adjacent soft tissues, and thus, surgical resection should incorporate the muscles of mastication (Masticator Space) (mainly T4b disease) undertaken for adequate margins. However, more extensive disease and involvement of the infratemporal fossa or base of skull may lead to resections with margin positivity and failure to reconstruct the defect eventually jeopardizing the outcomes for these patients. Thus, feasibility of Resection may be considered a surrogate prognostic marker of survival.[2,3]

Head and Neck cancers pose a significant challenge to the operating surgeon since the resectability criteria are subjective. The following may be used as technical criteria to determine whether complete resection is feasible, in cases of Carcinoma Buccal Mucosa:[1]

1. Buccal Mucosa Primary with peritumoral edema with or without satellite nodules that extend above the Zygomatic arch

2. Tumors extending into the high infratemporal fossa, with extension above the sigmoid notch of the Mandible[4]

3. Extensive skin infiltration, where achieving negative margins is difficult.

Induction chemotherapy had been developed as an adjunct to improve resectability for Borderline Resectable disease. It also allows for organ preservation in para mandibular disease and extensive diseases of the tongue. The survival advantage is estimated at 30%, by extrapolated data from prior studies. These regimes may be a two-drug or a three-drug regime (Docetaxel, Cisplatin and 5 Fluoro-uracil v/s Docetaxel and Cisplatin), with their share of clinical and financial liabilities.[1,5,6] 70–80% of patients respond favorably, with improvement in resectability of disease, with conversion from Borderline Resectable to Resectable.[1,7] Both Licitria[7] and Zhong[8] et al. described significant local control with chemotherapy prior to surgery. From the Indian perspective, VM Patil et al.[1] conducted a study at the Tata Memorial Hospital, Mumbai, where 721 patients of Stage IV Oral Cancer were enrolled and 43% had a significant size reduction and underwent surgery after 2 cycles of Induction Chemotherapy and the overall survival was 19.6 months. The multicenter National Cancer Grid-NACT Mandible study by Chaukar et al.[9] achieved 47% mandibular preservation with acceptable toxicity with no compromise of survival in patients with para-mandibular disease.

Artificial intelligence (AI)-based models are gradually gaining importance but have been limited in their usage in the medical domain. Studies to evaluate their utility in determining patient outcomes have been scarce. Machine Learning Models are generally employed by data scientists as problem-solving tools; however, their integration into the medical field has only been recent. CNNs, a form of deep learning within the realm of machine learning, is a complex computer-generated artificial intelligence algorithm that excels in tasks like image analysis, particularly in radiology. By automatically learning features from existing data through layers like convolutions and pooling, they provide a powerful tool for understanding complex patterns in medical images.[10]

Machine learning is increasingly being applied in head and neck oncology to support prediction and clinical decision-making. It has shown promise in forecasting treatment outcomes and enabling personalized therapy planning.[11] Models have been developed to predict distant metastasis,[12] lymph node involvement in early-stage oral cancer,[13] and locoregional recurrence using radiomics.[14] Survival outcomes in oral cavity squamous cell carcinoma have also been modeled effectively,[15] along with perineural invasion risk[16] and recurrence prediction.[17] Algorithms have been used to estimate treatment response to chemoradiotherapy[18] and to assess recurrence in laryngeal cancer.[19] Reviews have summarized various machine learning methods, datasets, and challenges in this domain.[20] Notably, a deep learning model using FDG-PET/CT imaging has shown accuracy in predicting malignancy and survival,[21] while radiomic features from imaging have demonstrated value in predicting chemoradiation response.[22]

Financial liabilities and resource limitations may cause some patients with advanced cancer to present to the outpatient department without prior testing. Inadequate image quality and disparities in CECT reporting delay surgery, thus negatively impacting the survival of these patients. Reporting oncological CECT scans requires meticulousness as it involves analyzing multiple images to identify whether a tumor can be surgically removed or if it is on the Borderline Resectable, based on the involvement of the High Infratemporal Fossa. The evaluation of clinical and radiological data is crucial, emphasizing the significance of reviewing CECT images to reduce inconsistencies. To tackle these difficulties, our Image classifier and Deep Learning Model was created. This model identifies patients who may gain advantages from induction chemotherapy, particularly those considered to have Borderline Resectable disease. This concept aims to enhance patient care and simplify treatment decisions in complicated cancer cases by combining new technologies with clinical practice, which has never been attempted previously. However, this programming model is limited by the following:

1. It does not include clinical data like the images of skin involvement and focuses on a limited cohort of Head and Neck cancer patients only.

2. It does not take Tumor Biology (Grade) into account, which may also affect the prognosis.

3. It requires preformatting of definite slices of a CECT scan which must be integrated into predetermined format of four images in the jpeg file which can only be analyzed by the model.

4. The model presently can be executed on a local server only and cannot be utilized on other platforms, which makes the use cumbersome.

Further similar endeavors or advances in the present algorithm to increase its versatility to other platforms may be pivotal in the future.

CONCLUSION

This novel artificial intelligence-based Image classifier addresses the concern of reporting discrepancy and provides a uniform decision-making framework for the treating surgical team. It offers important insights regarding upfront resectability and eligibility, for Induction Chemotherapy, thus potentially serving as a vital component in strengthening patient care within the context of Multidisciplinary Oncology.

Ethical approval

This study was conducted after Institutional Ethical Clearance in accordance with the ethical standards of the responsible committee on human experimentation (institutional or regional) and with the Helsinki Declaration of 1975, as revised in 2013. The letter of approval of this study is available to the editors for review.

Conflicts of interest

There are no conflicts of interest.

Funding Statement

Nil.