Leveraging data-driven insights for esophageal and gastric cancer diagnosis

Abstract

Background

Oesophago-Gastric Cancer (OGC) and its precursor, high-grade dysplasia (HGD), provide a significant health concern owing to their rapid development and often late diagnosis. Timely identification of individuals is crucial to avert illness advancement and enhance clinical results. Nonetheless, there has been little research on machine learning models and effective statistical techniques for predicting OGC risk.

Aims

This research seeks to uncover risk variables linked to HGD and OGC and to assess the prediction efficacy of several machine learning models using diverse patient data.

Methods

This research used a comprehensive methodology that integrates specialist analysis, statistical modeling, and data visualization. Thorough preprocessing and risk factor identification were conducted for the creation of regression models to forecast OGC and HGD outcomes. Moreover, visual analytics and interactive dashboards were used to improve interpretability and clinical significance.

Results

The evaluation of the models revealed that the linear regression model exhibited higher performance, with a Mean Squared Error (MSE) of 0.0244, a Mean Absolute Error (MAE) of 0.116, and an R-squared value of 0.046.

Conclusion

Our results demonstrate the efficacy of machine learning models, namely linear regression, as a dependable instrument for the early risk assessment of OGC and HGD. The integration of predictive modeling, statistical analysis, and dashboarding provides a robust framework that assists healthcare practitioners in diagnosis, treatment, and patient monitoring.

Introduction

Esophageal (OC) and stomach (GC) cancers are referred to as ‘cancers of unmet need’ because they are often diagnosed at a late stage and have low survival rates [1]. Trying to find out what this disease is in the early stages is not easy. These people have a 15% chance of living for 5 years; it is the sixth leading cause of death in the world [2]. Research has shown that chronic risk factors for gastroesophageal reflux disease (GERD) such as being overweight, smoking, excessive alcohol consumption, poor diet, and genetic predisposition—significantly reduce survival rates [3]. High-grade dysplasia is considered the earliest sign of esophageal and stomach cancer [4]. When cells in the esophagus or stomach undergo abnormal changes, they can develop into cancer if left untreated [5]. Early diagnosis allows for quick action and stops cancer from spreading. Conditions such as chronic inflammation, Barrett’s oesophageal reflux, and H. pylori infections are likely to be responsible, and treatment is the only way to prevent it early and prevent it from happening again. Pathologists look at tissue samples from endoscopic and incisional biopsies to find HGD [6]. It’s challenging to diagnose and treat HGD correctly [7]. After diagnosis, patients may require treatment to prevent dangerous growths from becoming deadly. Understanding HGD is crucial as it serves as a target for preventing esophagus and stomach cancer. Patients could be healthier and less likely to get sick or die if they got an evaluation and treatment quickly [8]. By looking into the molecular and cellular changes that happen during HGD, scientists and doctors may be able to predict better and treat this precancerous illness, which would lower its impact and the number of people who get cancer. The esophageal cancer and HGD study uses dashboards and data displays to help people understand and make decisions [9]. Oesophagus gastric cancer and HGD are therapeutic situations that affect patient health and well-being, but assessing and interpreting all the data may be challenging [10]. Not many research have examined predictive modeling in a thorough and integrated framework, despite the fact that many have concentrated on clinical, pathological, and risk factor evaluations of OGC and HGD. Current research mostly uses conventional statistical methods with little assistance from interactive tools or machine learning techniques. Furthermore, the lack of visual analytics and real-time integration limits the interpretability and therapeutic usefulness of the majority of models. Our method bridges these gaps by utilizing machine learning approaches, dynamic dashboarding, and regression-based predictive modeling to provide a scalable and interpretable solution. This work overcomes present limits in clinical decision assistance and extends previous techniques by combining statistical accuracy with easily navigable graphics.

Gastric cancer originates in the stomach lining and has the potential to spread to other regions of the body [11]. It is more prevalent among men in developing countries [12]. Estrogen in women provides a protective effect against this disease [13]. Smoking and alcohol use increase the risk factor for gastric cancer. OGC resulting from smoking often manifests in the upper region of the stomach near to the oesophagus [14]. Cigarette smoking is associated with a reduction in circulating epidermal growth factor and an increase in free radical production within the gastric mucosa [15, 16]. Tobacco use and H. pylori infection were identified as the major risk factors for peptic ulcer disease (PUD) in Danish adults, based on research including 2,416 individuals done in Denmark [17].

Current efforts focus on early detection as well as molecular mechanisms and outcomes related to therapy outcomes [18]. Comprehensive data analysis support is vital in aiding understanding, prevention, and management efforts given its wide-ranging effects both individually and on healthcare systems worldwide [19, 20]. With so much data from clinical, genomic, and imaging sources requiring our attention, it can be challenging to combine, manage, and display it effectively [21]. Furthermore, including contextual factors like patient demographics or curing journeys in visualizations is challenging. Achieving inclusive understanding requires careful thought. Planning intuitive visualization procedures that are seamless and accommodate diverse user foundations remains a progressing challenge [22]. Integrating predictive models consistently into visualization dashboards for real-time investigation and decision support remains unsolved [23]. Comprehensive evaluation frameworks are necessary to measure visualizations’ efficacy, convenience, and effect on insights and decision-making [24]. Ethical considerations related to data privacy and regulation compliance need to be carefully considered [25]. Addressing these challenges and filling knowledge gaps will advance Esophagus-Gastric Cancer/HGD research efforts, driving upgraded understanding, enhanced decision-making abilities, and potentially improved patient results. In light of these challenges, the main contributions of our studies are as follows:

• Development and evaluation of multiple machine learning models for predicting the risk of OGC and HGD.

• Identification of demographic risk and key clinical factors that contribute to OGC and HGD progression.

• Introducing a comprehensive strategy that combines visual analytics, machine learning, and statistical models to improve interpretability and clinical relevance.

• Development and implementation of interactive dashboards to help researchers and clinicians.

• For selecting the best prediction models, a comparative performance study utilizing Mean Squared Error (MSE), Mean Absolute Error (MAE) and R-squared is provided.

Significance

Esophagus-gastric cancer and high-grade dysplasia are topics that need investigation in depth [23], and this study fills that need. As far as we know, no research has yet found the optimum regression model for making predictions using HGD data through visual interpretation. We used the performance metrics of R-squared, mean absolute error (MAE), and mean squared error (MSE) to choose the best model. The best model for academic and professional prediction is the one with the lowest metric values.

The rest of the paper is organized as follows. Section 2 provides a background study, and Sect. 3 presents the proposed methodology, including Data collection, statistical analysis, machine learning models, etc. Section 4 discusses the result along with a comprehensive analysis, and finally, Sect. 5 concludes this study.

Background study

Esophageal Cancer [26] refers to malignancies affecting both the esophagus and stomach that show as multifaceted malignancies with different subtypes [27]. HGD is a precancerous condition associated with an increased risk of cancer. When discussing esophageal and gastric cancer it’s crucial to keep in mind its main characteristic: abnormal cell changes in tissues. As one form of Esophagus-Gastric Cancer HGD refers to non-normalized changes within cells lining either your esophagus or stomach that have the potential for transformation into malignancies if left untreated [5]. High-grade dysplasia (HGD) in gastroenterological pathology is often related to conditions like Esophageal Barrett’s, where acid reflux causes abnormal changes to occur on cells that line the lower portion of esophagus linings, leading to abnormal cell changes. HGD in Esophageal Barrett may indicate an expanded risk for creating esophageal adenocarcinoma; HGD refers to significant changes observed during Pap smears or cervical biopsies that indicate significant cell changes that require further investigation. These untreated cervical changes could increase your risk for cervical cancer; high-grade dysplasia has more serious and likely cancer-forming changes than its low-grade counterparts, so healthcare professionals monitor HGD closely; appropriate interventions such as surgery or surveillance may often be recommended to stop its progress into full-invasive cancer. OGC, also called esophagogastric carcinoma, refers to malignancies that form at the junction between the esophagus (food pipe) and stomach. This form of cancer may involve both the lower portions of the esophagus as well as the upper stomach tissue. Digestive disorders present a serious public health threat with potentially severe effects, as they interfere with normal digestive tract functioning and may result in symptoms including difficulty eating and weight loss, digestive discomfort, or diarrhea. OGC often presents itself at a progressed stage due to its subtle early side effects; detection therefore becomes an uphill struggle. Cancer Research United Kingdom reports that frequency rates for esophagus gastric cancer shift over regions and populations, being most frequent among older individuals aged 60 years or over. Lifestyle factors, including smoking, excessive alcohol intake, and obesity have all been associated with increased risks for this form of cancer. Gastroesophageal Reflux Disease (GERD), in which stomach acid often escapes through its production site and into the esophagus from where it was originally produced in the stomach, has also been identified as an independent risk factor for cancers of both the esophagus and stomach. Treatment options for esophagus-gastric cancer depend on various factors, including its stage, individual health history, and location of the tumor. Possible therapies could include surgery, chemotherapy, or radiation therapies as well as targeted therapies; early diagnosis plays a crucial part in improving prognosis and outcomes for those diagnosed with such cancers.

HGD is a widely researched topic, as shown in Fig. 1. Esophageal gastric cancer prevalence varies geographically, with higher rates found in Asia, Eastern Europe, and South America [28], as shown in Fig. 2. Cancer of the stomach may start in the lining of the stomach and metastasize to other parts of the body [11]. Men are more prone to this disease than women [12]. The disease is protected against in women by estrogen [13]. Gastric cancer risk factors include smoking and alcohol use. Tobacco-induced GC often appears in the upper part of the stomach, near to the oesophagus [14]. Al-Emran et al. listed dysphagia, involuntary weight loss, and chest discomfort as possible symptoms [29]. The predominant symptoms of oesophageal cancer are dysphagia, thoracic discomfort, weight reduction, dysphonia, persistent cough, emesis, osseous pain, and hemorrhage into the oesophagus [30]. Diagnosing requires various procedures such as endoscopy, biopsy, imaging studies, and staging [31]. Treatment options depend on the stage and location of a tumor and could include surgery, chemotherapy, radiation therapy, targeted therapy, or immunotherapy [32]. Oesophageal gastric cancer presents many unique challenges due to its aggressiveness and limited treatment options for advanced cases. Research efforts are critical to early detection, molecular mechanisms, therapeutic targets, and outcomes of treatment options available to advanced cases [33]. Visual representation: Patient records, genetic sequencing, imaging, and research create huge volumes of complex data [34]. Using data visualization, researchers can discover patterns, trends, and outliers in several sources [35]. Data visualizations assist academics, physicians, and policymakers in explaining and engaging with research findings [36]. Visualizing complex data quickly promotes collaboration and cross-disciplinary understanding. Dashboards enable stakeholder collaboration and informed decision-making [37]. Researchers and pathologists make educated judgments using visualization of healthcare data [38]. Data visualization helps researchers identify patterns, connections, and links to inform ideas and study methods. Pathologists may concurrently see patient health data, treatment outcomes, and prediction models using data visualizations to improve diagnosis, treatment, and care [39]. Data visualization aids healthcare decision-making [40]. Visualizing individual health data helps patients comprehend, monitor, and choose treatments. Physician-patient communication improves by visualizing symptoms, assessments, and treatment preferences [41]. Data visualization and dashboarding optimize key performance metrics and encourage high-quality improvement [42]. Visualizing healthcare data helps discover inefficiencies, bottlenecks, and variances that impede operations, enhance patient safety, and improve results [43].

Fig. 1.

High-grade dysplasia publication count in the database “PubMed” for the last 50 years

Fig. 2.

Esophageal gastric cancer search count around the globe since 2004

OGC data visualization: OGC and HGD data cover imaging, genomic, and clinical fields, which makes it challenging to interpret, organize, and visualize [44]. One of the biggest issues is developing user-friendly interactive visualization methods for different user backgrounds. Visualizing patient demographics, co-morbidities, and treatment routes is essential to illness comprehension [45]. Complex technology and outcomes that are hard to understand make it difficult for predictive models to incorporate them into real-time analysis and decision support [46]. To assess appropriateness, convenience, clinical outcomes, and decision-making, robust assessment frameworks are needed [47]. Data visualizations for patient healthcare must incorporate data privacy, security, and informed consent [48]. Addressing ethical issues in OGC/HGD visualization research will improve understanding, decision-making, and patient outcomes, leading to more productive research programs that advance this field and improve patient care. Current efforts focus on early detection, molecular mechanisms, and outcomes related to therapy outcomes [49]. AI models are widely used in the medical field for performing complex tasks [50]. Ali et al. proposed a novel method for the classification of cancer subtypes. They had to face the problem of data imbalance and high dimensionality of data. They used Cat Swarm Optimization (CSO) for feature selection and support vector machines for data classification. they used k-means clustering to extract features from the high-dimensionality dataset. K-means is used with CSO for determining the optimal clusters that can be used for solving the problem. Without using a feature extraction method, they achieved an accuracy of 81% with SVM, and after using the proposed feature extraction method, they attained 100% accuracy [51]. Another study depicts the methods for improving feature extraction. The study presents the current challenges in the omic data and the challenges that have been resolved by the researchers. They show the best methods that have been used so far. Authors emphasized that ML and DL are promising approaches for dealing with omic data. Authors suggest that there is a need to focus more on the DL domain [52]. Another optimization technique is presented by El-kenawy et al. The authors presented a novel approach, a nature-inspired optimizer called the Greylag Goose Optimization (GGO) algorithm. It is also a swarm-based optimization method. The method achieved good performance according to one-way analysis of variance (ANOVA) [53].

Methodology

In this research study, we analyzed data from the National Center for Audit of Oesophago-gastric Cancer. Machine and deep learning algorithms have brought a breakthrough in the medical domain [54]. We conducted an in-depth statistical analysis and comparative study using machine learning algorithms to determine which methods practitioners can use to manage large datasets and identify optimal strategies. Gastric cancer originates in the stomach lining and has the potential to spread to other regions of the body [11]. Though tissue regeneration is possible [55], it is still a fatal disease. the detailed investigation in this study will help practitioners predict OGC at an early stage. The investigation workflow is presented in Fig. 3. The methodology of the research paper is elaborated stepwise.

• Download the NOGCA dataset from the authentic data repository.

• Select the data from the years 2017 to 2021.

• Analyze the data statistically to gain data insights.

  • Endoscopic Treatment Data Analysis
  • Regional Statistics
  • Surveillance
  • Active Treatment Data Analysis
  • Average Success Rate

• Discovering patterns and deriving insights

• Comparative Analysis using ML models. The following models are used for the comparative analysis.

  • Linear Regression
  • Random Forest
  • Lasso
  • Decision Tree
  • Ridge
  • Gradient Boosting

Fig. 3.

Workflow of this study

Statistical analysis and machine learning algorithms reveal that one approach outperforms the others for this dataset.

Data collection

The National Oesophago-Gastric Cancer Audit (NOGCA) dataset [56] contains all clinical and statistical data on oesophageal-gastric cancer patients. This nationwide UK database collects data from healthcare organizations and medical institutes. This dataset contains many patient cases from 2017 to 2021, making it a useful EGC/HGD research and analysis data source. NOGCA provides important patient and therapy history factors such as age, gender, race, and location. Cancer pathological data includes tumor stage, histological subtype (adenocarcinoma and squamous cell carcinoma), grade, and anatomical location (oesophagus and stomach). Treatment information includes surgical type, chemotherapy, radiation therapy, targeted therapy, etc. The NOGCA dataset features various characteristics relating to each case, such as:

• Statistical Data: This covers patient-specific details like their age, sex, and other relevant statistical information.

• Pathological Stage Data: Knowing the stage at diagnosis makes a difference in understanding its progression further.

• Treatment Details: Details about the type of curative surgery treatments patients receive are also provided here.

• Death and Survival Rates: Provided here is data pertaining to 30-day adjusted mortality and 1-year survival rates respectively.

• Length of Patient Stay: The median of the length patient stays in the hospital while receiving cancer treatment.

• Tumor Records Submitted: Records submitted for tumor diagnosis and staging respectively.

• Patient Health Results: Information regarding patients’ status at discharge, pathological reports, and staging statuses like tumor, node, and metastasis (TNM).

• Case Ascertainment Rate (CARP): This measure indicates how often cases were properly ascertained during an audit.

The NOGCA dataset is known for its large test measure and longitudinal nature. A detailed investigation of esophageal gastric cancer and HGD wordly patterns is possible. The extensive and shifting information allows analysts to study the disease’s risk factors, treatment effectiveness, access to healthcare, and regional variance in occurrence. Security and privacy are crucial for any large health dataset. NOGCA data is anonymized, subject to strict privacy regulations to protect patient anonymity, and researchers must meet NOGA ethical requirements for the study. Researchers utilizing it for other research must get permission. Finally, the NOGCA dataset provides healthcare researchers and professionals with vital insights about oesophageal-gastric cancer and HGD that might enhance patient treatment and outcomes. NOGCA collects data from UK hospitals and clinics that diagnose and treat esophageal and gastric cancer. NOGCAs National Oesophago-Gastric Cancer Audit program tracks OGC patient treatment. “Data.gov.uk” is an open data site where the UK government distributes datasets across many places to make them available to the public, academics, and enterprises. The NOGCA dataset on “data.gov.uk” enables openness, enhances data-driven research and analysis of English E. and English G. cancer care, and aids data-driven decision-making. This information may help researchers, policymakers, and healthcare specialists identify cancer care quality patterns and make educated healthcare choices to enhance patient outcomes and services. The UK government fosters open data, researcher-policymaker cooperation, and evidence-based policymaking by uploading datasets like NOGCA on “data.gov.uk”. Through “data.gov.uk” stakeholders like NOGCA users may get information and enhance cancer care plans and treatment options.

Figure 4 provides an extensive snapshot of HGD cases across locations, covering key aspects like treatment, diagnosis and case numbers across each. Card, one depicts an average active treatment rate which stands at 0.8170 per case on an annualized basis, this represents the percentage or rate at which patients engage in active care over no active treatment for each dataset. A box plot highlights any outliers related to active vs no active care as outlier columns; North Central London and Northeast London show outlier values within “No Active Care”, suggesting unexpected data points within these regions that require further examination. A tree map displays confirmed diagnoses by second pathologists across various regions. This allows practitioners to see how often second pathologists confirm initial diagnoses as well as regional variance in this process of confirmation. This table shows the total number of High-Grade Dysplasia (HGD) cases by locations. Data visualization and analysis demonstrate that Northern has the highest case count at 167 while “West Yorkshire” had 13; these figures provide valuable insight into HGD prevalence across different locations. To deal with the missing values and data anomalies, those values were removed.

Fig. 4.

Dashboard HGD cases

Statistical analysis

In order to perform the statistical analysis, we used Tableau software [57]. Fig. 5 shows a visual presentation of how high-grade dysplasia (HGD) cases were distributed among cancer alliances during two different years, 2017 and 2021, and by measure of cut in the pie chart representing rate contributions by each cancer alliance to overall HGD cases. In 2017, the “Northern” Cancer Alliance saw the highest percentage of HGD cases at 12.8% of total cases; this indicates a greater concentration in this area during that year. Meanwhile, “West Yorkshire and Harrogate” cancer alliance saw only 8 cases total and thus comprised the smallest portion in its pie chart, showing this region had relatively fewer HGD patients overall in 2017. A notable shift occurred in 2021: When Northern Cancer Alliance became responsible for most HGD cases, increasing from 15% in 2020 to 17% by that year, only reported 6 cases - perhaps suggesting that over time this distribution will shift across different locations.

Fig. 5.

Shows the comparison of HGD cases from 2017 to 2021

Regional statistics

As seen in Fig. 6, regional statistics provide a thorough breakdown of 3 treatment approaches—active treatment, endoscopic treatment, and surveillance treatment—over different districts. As shown in Table 1, around 79% of cases receive active therapy on average, representing its prominence within our approach portfolio. The median is slightly below 80%, indicating there are moderate levels of variance around the mean. Even at 39.28%, its minimum percentage indicates relatively high variability levels. Maximum values of 100% indicate regions that exclusively choose active treatment; 0.12 standard deviation indicates relatively few points being scattered around the mean, suggesting consistency within active treatment percentage levels.

Fig. 6.

Regional statistics of HGD

Table 1.

Regional statistics

Statistics	Active Treatment	Endoscopic treatment	Surveillance
Mean	0.79	0.718	0.12
Median	0.78	0.736	0.11
Minimum	0.39	0.211	0.0
Maximum	1.0	0.95	0.39
Standard Deviation	0.1277	0.1657	0.098

Endoscopic Treatment Moving onto Endoscopic Treatment, Table 1 displays that on average approximately 71.80% of cases were managed endoscopically; its median value being close to 73.61% indicates this was relatively balanced across regions, though minimum levels at 21.10 were found in some regions to represent smaller proportions being treated endoscopically whereas 95.24% indicated it as being predominant approach in certain locations and its standard deviation value indicated more spread-out distribution reflecting greater variations among its use than with Active treatment approaches.

Surveillance It stands at 12.01%, demonstrating that around 12% of cases are treated through observation. The median value is 11.1%, indicating slightly right-skewed dissemination, while 0% as the least value means few regions do not depend on observation as a treatment approach, as in Table 1. Furthermore, 39.28% as a maximum value shows how surveillance plays a more significant part; its standard deviation esteem of 0.09 shows generally direct variations.

Active treatment and surveillance by the Cancer Alliance This study tracks the Cancer Alliance’s Total Active Treatment and Total Surveillance statistics. Normal monitoring decreases by 1.22 units per unit of Active Treatment across all entities (Fig. 7). However, Lancashire and South Cumbria have a higher surveillance involvement rate than active treatment at 24% vs 7%. Active treatment values in Southeast London are unusual; examining Active treatment and Surveillance might reveal regional concentration and dispersion. The largest proportion of active treatment patients is in Southeast London (100%), South Yorkshire and Bassetlaw (94%), and Kent and Medway (93%). They live in London (100), Yorkshire (94%) Medway (93%), and Kent (93%). North Central London had an Active Treatment rate of 68% to 100% in Southeast London, a 32% difference with an average of 81%. Five substances accounted for 29% of Active treatments.

Fig. 7.

Active treatment and surveillance by region

The total proportion of patients under surveillance across all entities is 2.13%, with Lancashire and South Cumbria contributing 24%, Humber Coast Vale 21%, and Northern 21%. Surveillance values are absent for 15% of entities, whereas 50% of entities account for 81% of surveillance utilization. Lancashire and South Cumbria provide 24%, more than double the total of 20 entities. In Fig. 7, groupings with comparable Surveillance values exhibit a clear pattern: 8 entities decline between 15 and 24%, while 5 others slide between 0 and 3%. These findings show changing approaches to active treatment and surveillance across regions, and the concentration of metrics within certain entities and significant variance in percentages show how important regional considerations are when analyzing healthcare patterns for health evaluation.

Average success rate by 2021 Figure 7 shows an area map of Active, Endoscopic, and Surveillance Treatment techniques by region for 2017 and their prevalence and effectiveness rates. Figure 8 reveals that Somerset and Wiltshire had the most active treatment cases at 89.62%, indicating a focus on vigorous care. North Central London had the fewest active treatment instances (67.86%), suggesting this technique should be minimized. Comparisons of 2021 success rates show this development. Somerset and Wiltshire had the greatest active treatment percentage in 2017 but now has a 0.90% success rate, suggesting its treatment efforts were effective. North Central London had the lowest active treatment percentage but a 0.68% success rate, indicating therapeutic progress. Somerset and Wiltshire had 0.88% endoscopic success rates, suggesting future growth. Endoscopic therapy seems to have worked in this location.

Fig. 8.

Statistical analysis

Endoscopic therapy had the lowest success rate in Great Manchester at 0.51%, suggesting obstacles or geographical differences. With a 0.24% surveillance treatment success rate, Lancashire seems to be successful in identifying and controlling cases. Somerset and Wiltshire had the lowest surveillance success rates at 0.3%, indicating room for improvement. It shows how objectives and results vary and how healthcare methods affect patient health. At this depth of examination, we undertake an in-depth data-driven journey, digging profound into critical metrics that provide invaluable information in the healthcare domain. Through rigorous examination, our aim is to understand the relationships and patterns these metrics hold which may provide new insight into aspects such as patient care strategies and results (Fig. 8 shows the Walle’s Distribution of Cancer Alliance Regions and Fig. 9 shows, the distribution of OGC cases by region). Figure 10 depicts our analysis of Total Cases with OES long margin across various periods and NHS Trusts, highlighting the significant disparity in performance between University Hospital Southampton and Portsmouth Hospitals, with University Hospital Southampton averaging 0.12 higher than Portsmouth Hospitals NHS Trust. Regional trends were apparent as Cancer Alliances and Welsh Regions focused on metrics, while the OES circum margin exhibited similar patterns to previous metrics. However, total cases for GAST Long Margin revealed significant discrepancies, particularly in the West Midlands, which displayed markedly divergent scores across entities, highlighting its considerable impact.

Fig. 9.

Distribution of OGC cases by region

Fig. 10.

Completion in OGC, Margins and nodes examination

Insights from total cases for OES long margin, in this comprehensive investigation, we explore the whole of total cases for OES long margin across different time periods and categorize it by NHS Trust & Health Board name as illustrated in Fig. 11. University Hospital Southampton NHS Foundation Trust often showed higher values compared to Portsmouth Hospitals NHS Trust, showing an average difference of 0.12. Wessex stood out as particularly impressive. One data point there stood out among others. On the other hand, Portsmouth Hospitals NHS Trust displayed lower values when compared with University Hospital Southampton NHS Foundation Trust with an average difference of 0.12; Wessex registered 0.88 in that instance. University Hospitals Dorset NHS Foundation Trust also posted lower figures than Portsmouth Hospitals NHS Trust with an average difference of 0.34; Wessex registered at 0.54 as evidence in this instance.

Fig. 11.

Completion in OGC, compete cases for OES and long margin

Discovering patterns in complete cases for nodes examined, moving our focus, we discover complete cases for nodes Examined by Cancer Alliance or Welsh Region and focus on their concentration among certain entities. As shown in Fig. 2, this number ranges between 2.36% to 9.39% across regions - West Midlands being notable at having achieved 9.39% surpassing even average; further demonstrating their role in shaping overall metrics landscape. This analysis underlines their significance and underscores key entities’ contribution.

Fig. 12.

Shows the complete cases for node examined

Cancer Alliance or Welsh Region, as a percentage of total cases examined, was further considered during the analysis, along with their Complete Cases Examined Rate expressed as a percentage. A total of 31.19 nodes examined across 23 entities was observed with North East London having 2.36% and West Midlands reaching 9.39% with North East. London falling between 2.36 and 9.39 respectively and being responsible for 7.04 of that difference; 35 entities contributed 56% each to total Nodes Examined with West Midlands surpassing it by one entity at 9.39; Additionally, the top three entities collectively comprised 26% overall nodes examined figures. Insights from complete cases for OES circum margin: Our analysis extends to Complete OES Circum Margin Cases categorized by Cancer Alliance or Welsh Region, reflecting trends seen in previous measurements. Figure 13 illustrates the significant role of major organizations, with East Midlands contributing 9.34%, in influencing this metrics environment. Similarly, our analysis included the assessment of the number of OES Circum Margin Complete Cases by Cancer Alliance or Welsh Region, culminating in a total of 30.09. This statistic exhibited cumulative values across 23 entities, amounting to 30.09. This distribution exhibited a notable range, from 2.24% (East of England - South) to 9.34% (East Midlands), indicating a total variance of 7.11%. Eight organizations, representing 35%, accounted for 56% of the total amount in this instance. The East Midlands, at 9.34%, surpassed all other entities by surpassing the norm by 7.96% points, as seen by prior measurements. Additionally, as previously noted, three entities together constituted 26% of the total Complete Cases for OES Circum Margin cases.

Fig. 13.

Complete cases for OES circum margin

Underneath patterns in complete cases for GAST long margin: Analysis focuses more closely on Complete Cases for Gast Long Margin (Fig. 14), showing a wide variety of results between 18% to 82% across entities, with West Midlands dominating with an enormous 63% difference; this highlights their influence in shaping the overall landscape. Last, this analysis examined Complete Cases of Gast Long Margin across Cancer Alliance or Welsh Region entities. Here, the cumulative total reached 3.61 for both entities with distribution patterns that ranged from 18% (Northeast London) to an impressive 82% (West Midlands), an approximate 63% difference. Furthermore, an impressive average sum of Complete Cases per entity registered 1.81 for Gast Long Margin cases.

Fig. 14.

Show the complete cases for Gast Long Margin

Driving insights from the patients with pathological records As part of our research, we examined Patient Records with Pathological Records across Cancer Alliances/Welsh Regions/NHS Trusts/Health Boards. Of all 5,121 total Patients With Pathological Records across these entities, West Midlands stood out with its impressive contribution of 521 Patients; their distribution varied between 19 and 572 entities, showing clear patterns within Patient records from different entities.

Understanding trends of patients with pathological Our exploration continues with the Patients With Pathological TNM Stage, showing the differences in this metric between Cancer Alliances or Welsh Regions. Figure 15 displays contributions by certain entities, specifically amongst the top three entities, which help shape this overall metric landscape.

Fig. 15.

Patients with pathology TNM stage

Discovering insights from the patients with nodal dissection As this study progresses, we explore Nodal Dissection over various entities. Nottingham University. Hospitals NHS Trust, Guy’s and St Thomas’ NHS Foundation Trust and Newcastle Upon Tyne Hospitals NHS Foundation Trust all make significant contributions that demonstrate healthcare provider impactful results. By further investigating, we discover the Northern region contributed 400 records across two subcategories for 7.81% of the overall dataset as seen in Fig. 16. Contributors mainly included Newcastle Upon Tyne Hospitals NHS Foundation Trust with 265 records, followed by South Tees Hospitals NHS Foundation Trust with 135. Subcategories ranged in number between 135 (South Tees Hospitals NHS Foundation Trust) and 265 (The Newcastle Upon Tyne Hospitals NHS Foundation Trust), yielding an average contribution to this region from this category (based on South Tees Hospitals’ share) of 200 contributions overall. Figure 17 depicts a KDE plot which shows the distribution of Nodal Dissection Metric, used for measuring medical-based nodal dissection severity. The graph depicts nodal dissection values within their range and their frequency across a dataset, while their shape displays how those node dissection values have been distributed throughout. The x-axis represents these ranges; its y-axis depicts their density. KDE plots show how nodal dissection values have been distributed throughout. KDEs often depict Somerset, Wiltshire, Avon and Gloucestershire where nodal dissection values most frequently occur with densities between 0.000 to 0.0016 on their y-axes reflecting likelihood levels.

Fig. 16.

Patients with nodal dissection

Fig. 17.

Distribution of patients with nodal dissection across different regions

Investigating trends in patients undergoing curative procedures Our analysis further explores patterns among Patients Undergoing Curative Procedures, providing insight into Cancer Alliances or Welsh Regions where Patients resided who had undergone curative procedures. West Midlands stands out with 11%, showing its notable influence; concentration among certain entities shows its critical position, as seen in Fig. 18.

Fig. 18.

Patients having curative procedure

Discovering patterns in patients with discharge data status We also investigate the connections between patients with discharge date and discharging status, providing insight into their interrelations. Figure 19 highlights trends and patterns that shed light on their significance as healthcare outcomes metrics; on this path, we propose to prepare healthcare professionals with knowledge for informed decision-making with positive patient healthcare results.

Fig. 19.

Shows patients with discharge status

Investigation of tumor record submission; Analysis variation by expected cases in different regions In this analysis, we study tumor records submitted within NCRAS/PEDW 2019–21 Expected Cases Analysis as part of Cancer Alliance or Welsh Region analyses and categorize them based on Cancer Alliance or Welsh Region designation. Analysis discovered that total tumor records submitted across nine entities amounted to 19,168. Notable contributions come from 150 to 199 tumor submissions with 4,964, 100–149 submissions totaling 4,137 submissions and 200–249 with 3,356. West Midlands led submissions in this range with 627, while Northern claimed 624 and North East London 351. Submissions varied between zero (0–49) submissions to 4,964 (150–199) for this time frame, making up an approximate variance of 4,868 with an average submission amount of 2,130. Tumor records submitted are concentrated, with three entities accounting for 65%. Meanwhile, those between 150 and 199 represent 26%. At 150–199, Tumor records submitted accounted for 4.964, representing 26% of total Tumor submissions across 20 subcategories across the nine entities. West Midlands is driven in submissions with 627 subcategory missions, followed by Northern with 624, and North East London with 351. There was an even distribution from Thames Valley (61 submissions) up to West Midlands (627), with an average submission count of 248.52. 8 subcategories made up 63% of cumulative submissions within this range; of those eight subcategories, two comprised percent. Analysis of tumor record submissions between 100 to 149 revealed 4,137 tumor records across 17 subcategories, representing 22% of overall submissions. Eminently, RM Partners West London driven with 541 submissions; followed by Somerset Wiltshire Avon Gloucestershire who contributed 446 and Greater Manchester with 430. Additionally there was an average distribution between North Central London 86 submissions to 541 (RM Partners West London), or approximately 243.45. Within this range, 7 subcategories accounted for 67% of submissions; 3 subcategories made up 34%. Last, our analysis explored submissions between 200 and 249; these submissions contained 3,356 Tumor records submitted across 12 subcategories for evaluation - making up 18% of submissions overall. Eminently, West Midlands led with 436 submissions; Northern had 428; Humber Coast & Vale had 409; Somerset Wiltshire Avon Gloucestershire had an average submission count of 279.67 within this range and five subcategories contributed 59% while two subcategories made up 26%, as seen in Fig. 20.

Fig. 20.

Variations among tumor records by region

Analysis key metrics across NHS trust/health records Figure 21 showcases 3 metrics, such as adjusted 30 days mortality rate, 90 days Mortality rate and Median Length of Stay (Days), across 35 NHS Trust/Health Board names. Studies of their relationships demonstrate that Adjusted 30 Days Mortality Rate does not have an evident relationship to the Median Length of Stay (Days). Although among NHS Trusts/Health Boards there may be certain entities which exhibit notable metric values as evidenced in Fig. 21, some entities do have noteworthy ratios. Gloucestershire Hospitals NHS Foundation Trust exhibits impressive values for both its 30-days mortality rate and 90-days mortality rate, both showing increased levels of death over time. Conversely, Royal Marsden NHS Foundation Trust and Swansea Bay University Health Board record lower values for Adjusted 90-Day Mortality Rate; with University Hospitals Coventry and Warwickshire NHS Trust reporting an elevated mortality rate. Norfolk and Norwich University Hospitals NHS Foundation Trust stands out with an average Adjusted 30-days Mortality Rate per Entity value of 0.01 while also boasting a lower Median Length of Stay (Days). Where the adjusted 90-days mortality rate per entity and median is 0.03 however the median length of stay per entity per entity was calculated at 10.51 days; hence this time-period had 10 deaths overall! As shown in Fig. 21, we examined three essential healthcare metrics - Adjusted 30-days Mortality Rate, 90-days Mortality Rate and Median Length of Stay (days). Our goal was to uncover any associations or patterns which might provide more insight into healthcare results.

Fig. 21.

Key metrics across NHS Trusts/Health boards

As we evaluate the relationship between Adjusted 30 day Mortality Rate, 90 Day Mortality Rate and Median Length of Stay (days), and Median Length of Stay (days), there was no strong and stable correlation. This indicates that changes to mortality rates over a certain timeframe might not directly influence hospital stays directly; while individual entities may exhibit higher values for certain metrics than others; our analysis seeks to uncover broad trends within our dataset by visual depicting its distribution - Adjusted 30 Day Mortality Rate ranges between 0.06 to 0.06 while Median Length of Stay (days), which ranges between 7 and 13.

Results and discussion

In this section, we will discuss the results and will analyze them in detail. For health care data prediction, we performed comparative analysis of regression models. We conducted an evaluation to compare different regression models capacities to foresee healthcare-related measurements, with an aim of selecting an ideal model based on given highlights.

Comparative analysis

Metrics utilized Mean Absolute Error (MAE), Mean Squared Error (MSE), and R-squared coefficient of determination as means for evaluation that gives an idea of the accuracy and explanatory power of each model [58]. At our tests we evaluated 6 regression models: Random Forest, Linear Regression, Ridge Regression, Lasso, Decision Tree, and Gradient Boosting. Each of their predictive capacities were then measured against 4 metrics and displayed in our Table 2 as results. In our evaluation of regression model performance metrics, we assessed different algorithms using Mean Absolute Error, Mean Squared Error and R-squared as performance measures for their algorithms. This enabled us to accurately gauge accuracy, precision and goodness-of-fit against our data source. With each metric assessed against our success criteria built up prior. Models evaluated included Random Forest Regression, Lasso Regression, Lasso Regression, Linear Regression Decision Tree Regression, Gradient Boosting Regression, and Ridge Regression.

Table 2.

Performance metrics comparison of regression models

Metric name	Linear	Random Forest	Lasso	Decision Tree	Ridge	Gradient Boosting
MSE	4.1e-33	0.0024	0.0078	0.0055	0.0001	0.0048
MAE	8.7e-17	0.0430	0.0748	0.0475	0.9709	0.0503
R-Squared	1.0	0.5117	0.6015	0.1253	0.9709	0.0045

According to the results shown in Table 2, which presents a comparison of performance metrics for various regression models based on their respective performance scores, the Ridge Regression model is found to be the most suitable for healthcare data. As is evident, the Ridge Regression model has achieved remarkable performance, as evidenced by its exceptionally low value of Mean Squared Error, which is approximately 0.0001, and its R-squared score, which is 0.97. These scores demonstrate the model’s remarkable level of success as an active treatment variables predictor, which is what makes this solution appealing to healthcare providers. The requirements of healthcare experts are satisfied by this model, which provides very precise predictions for the percentage of patients who will be successful in receiving active therapy. The low mean square error values indicate that the predictions generated by this model are in close agreement with the actual values, and the high R-squared score indicates that it captures the majority of the variation that is present in the variable that it is attempting to predict. In order to make informed decisions about the distribution of resources and the formulation of treatment strategies, experts in the healthcare industry may rely on the predictions made by the Ridge Regression model. More often than not, medical practitioners may depend on the high accuracy of this model to properly forecast the likelihood of success for active therapy for particular patients based on the input characteristics that are supplied. This, in turn, leads to more targeted, effective interventions, which ultimately enhance both patient healthcare results and the quality of care delivered.

All the models are related; the justification for using Mean Square Error is that it measures the average squared differences between predicted and actual values. Lower MSE indicates better model performance. Mean Absolute error (MAE) measures the average absolute difference between predicted and actual values. Lower MAE indicates better model performance. R-squared indicates how well the model’s predictions match the variability in the actual data. A higher R-squared indicates a better fit to the data.

Hyperparameter fine-tuning of data

One of the outcomes of the hyperparameter tuning operation that was performed in order to optimize machine learning models was the realization of improved generalization capacity and model efficiency. A newly modified model with hyperparameters that have been tweaked to increase generalization is depicted in Table 3. This is evidenced by lower MSE and MAE values for some models after tuning hyperparameters. On the other hand, R-squared values for some Gradient Boosting Regression models (such as these ones from Gradient Boosting Regression) appear to have worsened after hyperparameter tuning, which indicates that they may no longer fit their data accurately after hyperparameter optimization. When evaluating tuned models, the success criteria that were defined previously are used. These criteria include reducing mean squared error (MSE) and mean absolute error (MAE) while simultaneously maximizing R-squared values. Following the tuning process, both linear regression and Ridge Regression were able to attain high R-squared values. The proof that both models continue to be acceptable for this data set can be found in Fig. 22; for validation of this assertion, please refer back to Fig. 22.

Table 3.

Performance metrics (after hyperparameter fine tuning of model)

Metric name	Linear	Random Forest	Lasso	Decision Tree	Ridge	Gradient Boosting
MSE	2.9553e-20	0.0018	0.0046	0.0037	1.9289e-06	0.0099
MAE	3.6064e-15	0.0215	0.0493	0.0543	0.0023	0.0888
R-Squared	1.0	0.705	0.256	0.014	0.99	0.597

Fig. 22.

Three key metrics relation

The purpose of this study was to assess the efficacy of active treatment for HGD. After a number of models were evaluated, ridge regression performance was best. This model had a very low error and a high R-squared value, showing that it matched the data well and made accurate predictions. Ridge Regression helps to reduce the problem of features being too similar to each other, which makes predictions more reliable compared to Linear Regression. But be careful, it might still make predictions too perfect, which could cause problems. At the same time as its success is celebrated, however, it must also be recognized that Ridge Regression may only apply to some datasets due to varying characteristics of each. Even when showing positive results on one dataset, it might not apply in all instances.

Predictions by this model could also be affected by its limited dataset size and noise in healthcare data, yet its superiority lies with its regularization term that prevents overfitting by constraining feature coefficients. Ridge Regression’s suitability could come from its ability to generalize well to unpublished data - essential when used for healthcare prediction. Furthermore, its regularization process might help account for any correlated features present within medical datasets that need regularizing as part of regularization purposes. In summary, the Ridge Regression model was able to accurately predict the success of treatment for HGD patients using active treatment methods because it had a low MSE value and a high R-squared value. However, before using the model, it’s important to consider any limitations or characteristics of the dataset.

Predictions of the OGC patients with clinical stage (0–3)

Which regression models get exact projections for the extent of patients at clinical stage 0–3 who receive curative treatments plans using features like Cancer Alliance/Welsh Region, NHS Trust/Health Board code and All Patients? This question explores the predictive capabilities of regression models when used to ascertain whether patients in clinical stages 0–3 will receive curative treatments plans. Utilizing available features, we evaluated which regression model performed the best at performing this prediction task. This analysis can give healthcare providers insight into factors influencing treatment plans to develop efficient patient care strategies. Based on these metrics, we can suggest an ideal regression model to fit healthcare data sets; Linear Regression illustrated the lowest Mean Squared Error of 0.0244 (MSE), as shown in Fig. 23. These results illustrate its superior predictive accuracy over competing models and its ability to forecast values with minimal average deviation from reality. Although all models had fairly modest R-squared values, linear regression showed the highest R-squared score and so captured more variance from target variables than other approaches. Linear Regression’s robust performance across numerous metrics make it an attractive solution for healthcare practitioners. Prediction accuracy as evidenced by low Mean Square Error (MSE) and Mean Absolute Error (MAE) scores allows healthcare practitioners to make better informed decisions that result in enhanced patient outcomes. An R-squared score above 0.85 indicates a model’s ability to accurately interpret data patterns with greater interpretability and insights than ever. These models may assist practitioners when making evidence-based decisions for patient care strategy purposes or optimizing healthcare-related metrics in decision making situations.

Fig. 23.

Performance of tuned models

Hyper-parameter tuning of models (OGC)

Comparing regression model performance before and after hyperparameter tuning reveals only minor changes; however, tuning did not significantly improve their generalization, as shown in Tables 4 and 5. While Linear Regression, Ridge Regression, Lasso Regression, and Lasso Regression had minimal effects from tuning; their R-squared values remained fairly consistent after tuning; but Lasso Regression showed positive change, changing from having negative R-squared values into having positive values; suggesting this led to improved fit by way of tuning (Figs. 24 and 25), suggesting it lead to improved fit overall generalization for this model by tuning. Contrarily, models exhibiting initial bad fits such as Random Forest Regression, Gradient Boosting Regression, and Decision Tree Regression did not experience substantial improvements post tuning (Fig. 26 displays R-squared values that did not change significantly even with hyperparameter adjustments); Lasso Regression showed only minor adjustments post hyper-parameter tuning highlighting its role as just one component for optimizing models. Figure 25 results demonstrate how model selection and advanced techniques could play an instrumental role in increasing generalization for weak-fit models; hyperparameter tuning may positively influence certain specific models but might not provide sufficient improvement when applied to complex healthcare datasets.

Table 4.

OGC comparison of performance metrics of regression models

Metric name	Linear	Random Forest	Lasso	Decision Tree	Ridge	Gradient Boosting
MSE	0.0224	0.0229	0.0242	0.0310	0.0224	0.0229
MAE	0.1165	0.1143	0.1221	0.1351	0.1165	0.1125
R-Squared	0.0470	0.0253	0.0277	0.3171	0.0470	0.0272

Table 5.

OGC - Hyperparameter tuning of performance metrics

Model	MSE	MAE	R Squared
Linear Regression	0.0224	0.1165	0.0470
Lasso Regression	0.0221	0.1166	0.0603
Ridge Regression	0.0224	0.1165	0.0471
Random Forest Regression	0.0241	0.1179	−0.0214
Decision Tree Regression	0.0266	0.1214	−0.1288
Gradient Boosting Regression	0.0237	0.1102	−0.0085

Fig. 24.

OGC-mean squared error

Fig. 25.

OGC-mean absolute error

Fig. 26.

OGC R squared

Our selected research question looked to predict the number of patients with stage 0–3 cancer receiving curative treatment plans, using different regression models. Of all available regression models, Linear Regression proved the top performer with MSE of 0.0244, MAE 0.116 and R-squared of 0.046 as shown in Fig. 27; These measurements demonstrate how closely its predictions match real values despite of this, its R-squared value remains slightly negative. However, it’s basic to recognize few feedback of linear regression models such as R-squared that suggests their superior performance relative to others may actually capture less variance within data. This may be attributable to healthcare dataset complexity as well as potential nonlinear relationships not fully captured by models as well as specific features used for prediction that influence generalization of models. Even with its inherent limitations, Linear Regression emerged victorious among other models due to its versatile capacity to quickly iteratively adjust and enhanced predictions by targeting already misclassified instances. Due to this flexibility, this model better captured patterns within our data set making it the obvious choice for this prediction task.

Fig. 27.

The visualization of performance metrics

Model parameters

In order to provide transparency, we provide the model parameter details so the results can be reproduced. It includes default as well as customized values of parameters.

Linear Regression Default parameters.

Ridge Alpha 1.0 before tuning and tuned alpha ranges between 0.01 and 1.0.

Lasso Alpha 1.0 before tuning. Alpha ranges between 0.001 and 0.5 after hyper-tuning.

Decision TreeMaximum depth of the tree is set to None. Minimum sample split is set to 2, and minimum sample leaf is set to 1.

Random ForestFor Random Forest, n-estimators are set to 100, maximum depth is None, and maximum features are automatically selected by the model.

Gradient Boosting For Gradient Boosting, n-estimators are again set to 100, learning rate to 0.1, maximum depth to 3, and subsample to 1.0.

Clinical workflow integration

One of the primary strengths of our work is that the suggested model may be readily used in clinical settings. This research used a real-time dataset, which is applicable in clinical situations, hence rendering it very appropriate for real-time scenarios. It can be seamlessly integrated into the electronic health record system with little overhead. Clinicians may use the model’s continuous risk assessment inside decision-support dashboards to promptly identify high-risk patients, prioritize referrals, and customize follow-up measures. It can provide immediate predictions that can significantly assist a variety of practitioners.

Table 6 provides a comparison between the state-of-the-art methods and the methods proposed in this research study.

Table 6.

Comparison between the proposed method and state of the art methods

Author	Modality	Model Used	Evaluation metric
Ghatwary et al.	WLI	Faster R-CNN, SSD	sensitivity 96%
Wu et al. EAC	WLI	Faster-RCNN, DSN	Accuracy 96.28%
Groof et al.	WLI	hybrid ResNet-Unet	Accuracy 89%
Tang et al.	Not mentioned	DCNN	Accuracy 91.30%
Gong et al.	WLI	No-code deep learning tool“Neuro-T”	Accuracy 95.6%
Putten et al.	endoscopy images	U-Net, Transfer Learning	Accuracy 87.50%
Proposed model	CTD	Ridge, Linear, Decision Tree	MSE = 0.000001, R2 = 0.99.

Practical deployment aspects of the proposed model and user interface considerations are discussed in the following subsection.

Practical deployment aspects

The proposed predictive model, which is developed using structured clinical data (NOGCA), is intended for implementation in the real world inside hospital information systems and electronic health record (EHR) platforms. The integration process involves just a small amount of change to the processes that are currently in place since it is dependent on tabular patient data that is already gathered during normal care (for example, demographics, staging, and treatment records). In order to provide scalability and accessibility across a number of different clinical locations, the model may either be hosted on hospital servers or delivered using cloud-based clinical decision support (CDS) systems.

User interface considerations

In order to guarantee clinical usability, the predictions of the model have to be shown using a dashboard interface that is included in the electronic health record system. The user interface should include the following features.

• A visual representation of the risk score for each patient (for example, the possibility of benefiting from curative or active treatment).

• Physicians can model outcomes by using interactive filters such as age, stage, and therapy type.

• To prioritize treatment for high-risk patients, color-coded notifications should be provided.

• To facilitate case discussions and MDT (multidisciplinary team) sessions, downloadable summaries should be provided.

• It is essential that the interface should be user-friendly, consistent with clinical language, and require just a minimum amount of training to facilitate the smooth adoption of the system by oncologists, surgeons, and care coordinators.

• To ensure that patients who are most likely to benefit from curative therapies are given priority, it is important to optimize resource allocation.

In the long run, this may result in increased survival rates, higher quality of life, and more cost-effective care by lowering the amount of unnecessary therapy or missing chances for early intervention during treatment.

Conclusion

This work aims to contribute to health informatics and predictive analysis. To achieve this objective, we used regression-based modeling and the NOGCA dataset. The dataset is a real-time compilation that aids in achieving interpretability and transparency. We used several statistical analyses on the dataset. Our findings provide a system that is both replicable and interpretable, which may assist healthcare practitioners in data prediction. The proposed model’s simplicity and clarity make it suited for explainability-critical scenarios like interdisciplinary team meetings and patient-facing tools. One of our strengths is using real-world, high-quality NOGCA clinical data. This guarantees that the results are based in healthcare reality and can be applied to policy and practice.

Our in-depth exploration of the NOGCA dataset has produced invaluable information that holds vital implications for healthcare planning and decision-making, by rigorously pre-processing data to guarantee its integrity and set the stage for further analyses. Data visualization was an essential element in our process for uncovering hidden patterns, relationships, and trends within the NOGCA dataset; various plots and visualizations enabled us to uncover hidden relationships within this information as we created plots/visualizations which allowed us to unearth hidden connections among them, to provide comprehensive knowledge about the healthcare landscape as we identified variables which could potentially alter cancer treatment planning strategies as we tested methodology against rigorous technical evaluation standards. Predictive modeling, cross-validation, and hyperparameter tuning were employed in our quest to build predictive models capable of accurately estimating both active treatment patients and those who possessed stage 0–3 cancer and curative treatment plans; our methodology ensured both robustness and reliability for our models. We utilized various regression models to choose an approach with optimal predictive accuracy.

Linear Regression provided consistent performance, while Lasso and Ridge Regression introduced regularization strategies that enhanced generalization. Decision Tree, Random Forest models, as well as Gradient Boosting models, showed promise through parameter tuning; however, they succumbed due to overfitting, due to Gradient Boosting’s susceptibility. After extensive hyperparameter tuning our models exhibited enhanced performances concerning mean Squared Error (MSE) and Mean Absolute Error; thus improving predictive accuracy significantly. R-squared values were relatively consistent over the study’s course, suggesting that our models’ predictive power is closely tied to the complexity of the data behind them. Recognizing our study’s limitations is of critical importance. It may not cover all factors impacting cancer treatment planning, leading to possible oversimplification or unobserved variables compromising the predictive power of our models. Future work must count additional relevant features while exploring more progressed techniques like outfit strategies.

This study presents several limitations. In this study, we employed several machine learning methods; however, we did not conduct a comprehensive comparison of the proposed method with all available ML techniques. Secondly, we did not incorporate additional datasets. In the future, we will utilize additional datasets and collaborate with healthcare experts to validate results. We will add temporal trends and longitudinal patient data to the model may increase prediction power and patient trajectories. Future work could incorporate techniques such as RFE or SHAP-based explanations to complement this clinically driven feature selection process and enhance model interpretability.

Abbreviations

OCC

Oesophago-Gastric Cancer

HGD

High-grade dysplasia

MSE

Mean Squared Error

OC

Esophageal cancer

GC

Gastric cancer

EBD

Gastroesophageal reflux disease

PUD

Peptic ulcer disease

CSO

Cat Swarm Optimization

GGO

Greylag Goose Optimization

ANOVA

Analysis of variance

NOGCA

National Oesophago-Gastric Cancer Audit

TNM

Tumor Node Metastasis

CARP

Case Ascertainment Rate

EHR

Electronic health record

CDS

Clinical decision support

MDT

Multidisciplinary team

Author contributions

Shafaq Zahoor, Zaib Unnisa, and Sajid Iqbal contributed to data collection, preprocessing, and literature review. Irfan ud Din (corresponding author) designed the study methodology, developed the predictive models, and conducted the analysis. Roobaea Alroobaea and Oumaima Saidani were responsible for data visualization, results interpretation, and manuscript refinement. Shafaq Zahoor and Zaib Unnisa drafted the initial manuscript; Foong Law reviewed and edited the final version. All authors reviewed and approved the final manuscript.

Funding

Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2025R760), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia.

Data availability

In this study, we did not collect any data ourselves. Instead, we utilized the publicly available NOGCA dataset, accessible via the NOGCA website [59].

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interest

The authors declare no competing interests.