Improving Ovarian Cancer Subtyping with Computer Vision Models on Tiled Histopathological Images

Abstract

Ovarian cancer remains one of the most challenging cancers to diagnose due to its non-specific symptoms, lack of reliable screening tests, and the complexity of detecting abnormalities. Accurate subtype classification is crucial for personalised treatment and improved patient outcomes. In this study, we developed a machine learning pipeline fine-tuning pre-trained computer vision models to classify ovarian cancer subtypes from whole slide images (WSI). Using targeted tissue masks for necrosis, stroma, and tumour regions as a proof of concept, we demonstrated the efficacy of tiling masked regions to transform a complex detection-then-classification problem into a simpler classification task. Our method achieved high accuracy in tile-level classification, with a subsequent extension to subtype classification via majority voting on tiled images. Precision exceeds 90% across subtypes, which highlights the potential of scalable, automated systems to assist in ovarian cancer diagnostics. These findings contribute to the broader field of computational pathology, paving the way for enhanced diagnostic consistency and accessibility in clinical settings.

Introduction

Ovarian cancer, like many malignancies, is marked by the uncontrolled proliferation of cells, leading to the invasion and destruction of healthy ovarian tissue and potential metastasis. Diagnosis is particularly challenging due to its non-specific symptoms that often mimic benign conditions [1], the absence of reliable screening protocols [2], and the anatomical positioning of the ovaries deep within the pelvic cavity [3]. These factors contribute to delayed detection and poor prognoses, with some subtypes presenting a five-year survival rate as low as 28% [4]. Histopathological examination remains the gold standard for diagnosing ovarian cancer subtypes. However, this process is subject to inter-observer variability and diagnostic inconsistency [5]. These issues are exacerbated in regions with limited access to specialised pathologists or in health systems facing workforce shortages.

Computer vision techniques have been used to improve the diagnosis of various cancers [6, 7]. A computer vision classification model is a type of machine learning model designed to interpret and understand visual data from the world around us. These models are trained on images and videos related to their intended purpose and can be made to recognise patterns and objects, some of which may not even be distinguishable by the human eye. Li et al. [8] used computer vision models to diagnose pancreatic cancer from PET/CT images. Ziyambe et al. [9] used deep learning techniques for ovarian cancer classification in pre- and post-menopausal women. Kodipalli et al. [10] used a single pipeline to segment and classify ovarian tumours into the categories: benign and malignant. They did this via an image-to-image translation approach using a conditional generative adversarial network.

Ovarian cancer consists of several histological subtypes, each with distinct characteristics. Clear cell carcinoma (CC) is a relatively rare subtype of ovarian cancer, often associated with endometriosis, and is typically characterised by clear cells that form glandular structures [11]. Endometrioid carcinoma (EC) arises from the endometrial tissue and often exhibits glandular patterns resembling the normal endometrium [12]. This subtype is commonly associated with both endometriosis and certain genetic mutations. Healthy tissue serves as a baseline for comparison, with no signs of cancer present. High-grade serous carcinoma (HGSC) is the most common and aggressive ovarian cancer subtype, characterised by high-grade, poorly differentiated cells and rapid growth, often with a spread to the peritoneal cavity [13]. Low-grade serous carcinoma (LGSC) is a less common subtype that tends to grow more slowly than HGSC and is often associated with less aggressive behaviour and a better prognosis [14]. Mucinous carcinoma (MC) is characterised by mucus-producing tumour cells and tends to have a unique morphology, often presenting with large cystic structures and a more indolent growth pattern compared to other subtypes [15].

The presence of necrosis in ovarian tumours can indicate aggressive behaviour, as rapidly growing tumours tend to develop necrotic areas [16]. Pathologists may use the degree of necrosis as one factor in assessing the tumour’s aggressiveness and potential prognosis. In ovarian cancer, the tumour stroma can influence the tumour’s growth and spread [17]. For example, a dense or reactive stroma might promote tumour invasion into surrounding tissues, contributing to metastasis. Studying the characteristics of stroma, such as its cellular composition, can give insights into tumour behaviour and treatment response. Ovarian tumours may develop subtypes with specific molecular and cellular features, which can influence treatment decisions and prognosis [5]. Understanding the characteristics of the tumour itself, including its composition, cellular structure, and genetic profile, is crucial in determining the best approach to treatment.

Whole slide images (WSI) provide a comprehensive view of an entire tissue section. This facilitates the examination of the spatial distribution of cells, tumour morphology, and potential interactions within the tissue [18]. The heterogeniety of WSI allows for the observation of variations in cell morphology, density, and other features across the entire tissue section [19]. WSI is also the preferred type of image for subtyping or grading tumours in diagnostic tasks [20].

In this study, we fine-tune a pre-trained computer vision model using targeted masks that highlight necrosis, stroma, and tumour regions within whole slide images (WSIs), each categorised by ovarian cancer subtype. The absence of comprehensive annotation poses a challenge to achieving a complete understanding of overall tissue composition. To address this, we reframed the problem as a classification task, as opposed to a detection-then-classification task. By tiling the images and focusing on the classification of individual tiles, the need for a separate detection stage was eliminated. This approach utilises the available masks to train a targeted classifier, enabling precise classification at the tile level while streamlining the overall process. Furthermore, the overall classification of an image can be determined by aggregating the results from its tiles, with the majority class among the tiles representing the final classification for the image. This streamlined pipeline achieves over 90% precision through tile-level learning and majority voting, demonstrating strong performance even in annotation-sparse settings. Our findings support a scalable, clinically viable framework for ovarian cancer subtyping and contribute to broader efforts in precision diagnostics.

Method

Data

The dataset is compiled from multiple sources [21–24] and is available on Kaggle [22]. Whole slide images (WSIs) are provided at 20$\times$ magnification, with image dimensions ranging from 3000 $\times$ 3000 pixels to 100,000 $\times$ 50,000 pixels. The distribution of image sizes is illustrated in Fig. 1, while class distributions are summarised in Table 1. Representative samples are shown in Fig. 2.

Fig. 1.

Histogram representing number of images by size

Table 1.

Data distribution

Subtype	Count
HGSC	217
EC	119
CC	94
LGSC	42
MC	41

Fig. 2.

Samples of each cancer subtype

WSIs are accompanied by illustrative masks annotated by pathologists, highlighting tumour (red), stroma or healthy tissue (green), and necrosis (blue). These masks provide valuable guidance for downstream analysis, as non-zero values in the red, green, or blue channels indicate pixels of interest. However, it is important to acknowledge the substantial limitations of these masks. Their annotations are non-exhaustive and primarily illustrative, often omitting large regions that contain relevant tissue. As a result, some tumour, stroma, or necrotic regions may remain unlabelled, and additional tissue types not represented in the masks may be present. This incomplete coverage introduces the risk of mischaracterising tissue distributions and under-representing critical patterns. Such limitations highlight the need for analysis approaches that do not rely on the presence or quality of explicit masks, motivating the development of methods capable of robust inference directly from raw WSIs. A sample of these masks can be seen in Fig. 3.

Fig. 3.

Sample image with masks

In this study, the differentiation of cancer types is approached in stages, starting with a proof of concept by first classifying the three mask categories: necrosis, stroma, and tumour. This initial step lays the foundation for distinguishing cancer types based on the mask classifications. Figure 4 illustrates how the initial dataset for the mask classification was constructed from the data.

Fig. 4.

Building tiled tissue-type dataset from available masked regions

Model Architecture

There are multiple architectures fit for this purpose; however, the architecture of choice was EfficientNetB0 [25]. EfficientNetB0 facilitates an input size of 384 $\times$ 384. This tile size not only addresses the practical constraints posed by GPU memory limitations but also ensures compatibility with the dataset’s original structure. A few popular architectures similar in size were considered, such as resnet18 [26] and classification heads of YOLO [27], but early experimental results were poor. This architecture was trained using Pytorch 2.0 [28] and CUDA 11.7 on an Nvidia GPU 4090 with 24GB RAM.

Training Procedure

The dataset was evaluated using tenfold cross-validation, with each fold consisting of a 75% training split and a 25% testing split. Each training mini-batch had an even distribution of classes. This was made possible by oversampling the minority classes. To prevent information leakage due to spatial and semantic correlations between tiles from the same WSI, we performed all dataset splits at the WSI level. Specifically, tiles from a given slide were only included in a single split (training or test). During inference, tile-level predictions were aggregated to produce a slide-level prediction, as illustrated in Fig. 5. We used majority based pooling across all tiles from a slide to generate the final WSI-level classification.

Fig. 5.

Example of how inference is performed with tiling

Results and Discussion

The overall test accuracy of the model was $94.27\%\pm 0.79\%$, demonstrating its effectiveness in correctly identifying the mask type in the majority of cases. A breakdown of the confusion matrix is shown in Table 2. Notably, the model achieved near-perfect classification performance for tumour tiles, with only minimal misclassifications observed. This suggests that the morphological and textural features of tumour regions are distinctive enough for the model to learn and differentiate effectively. However, there were a few instances where necrosis and stroma tiles were misclassified as tumour or as each other. This confusion likely arises from visual similarities and overlapping histological features among these tissue types. For example, necrosis often occurs within or adjacent to tumour regions and may contain residual cellular structures

Table 2.

Confusion matrix for mask classification

Actual \Predicted	Necrosis	Stroma	Tumour
Necrosis	39	0	4
Stroma	0	306	24
Tumour	0	9	1608

or fragmented nuclei that resemble poorly differentiated tumour tissue. Similarly, stroma, when densely fibrotic or infiltrated by tumour cells, can exhibit architectural patterns that blur the boundary between stromal and malignant regions. These subtleties may be challenging for the model to resolve, especially in the absence of broader spatial context or in cases where tiles are extracted from transitional zones. Additionally, potential inconsistencies in annotation or class imbalance may further contribute to these misclassifications, highlighting opportunities to refine training data quality or introduce context-aware modelling strategies.

• Necrosis: This represents dead or decaying tissue often found in tumour regions. In the confusion matrix, 39 tiles were correctly classified as necrosis, with 4 misclassified as tumour, resulting in $90.7\%\pm 1.1\%$ precision.

• Stroma: This refers to the supportive tissue surrounding tumour cells. The model correctly classified 306 tiles as Stroma, with 24 misclassified as tumour, resulting in $92.7\%\pm 0.8\%$ precision.

• Tumour: This represents the cancerous tissue itself. The model accurately identified 1608 tiles as tumour, with 9 misclassified as stroma, resulting in $99.4\%\pm 0.1\%$ precision.

Tables 3 and 4 show the confusion matrix averaged across ten folds, along with more granular metrics like F1, precision, and recall. Figure 6 illustrates the normalised confusion matrix. The model achieved high classification performance across all six categories, with F1 scores exceeding 0.97 for each class. Particularly, strong results were observed for LGSC and CC, with F1 scores of 0.998 and 0.995 respectively. Most misclassifications occurred between HEALTHY and tumour classes, such as 0.78% of HGSC tiles being predicted as HEALTHY, and 0.54% of EC tiles as HEALTHY. These cases may reflect either genuine morphological overlap or limitations in the ground truth masks, which were not exhaustive and often omitted relevant tissue regions. The strong overall performance, despite these label imperfections, suggests the model is robust and capable of generalising well to histologically meaningful features. These findings also underscore the value of the proposed approach in mitigating challenges posed by weak supervision and incomplete annotations.

Table 3.

Confusion matrix for subtype classification

	CC	EC	HEALTHY	HGSC	LGSC	MC
CC	1206	0	4	0	0	0
EC	0	1668	9	2	0	2
HEALTHY	7	10	1786	17	1	8
HGSC	0	2	23	2930	0	0
LGSC	1	1	2	0	1114	0
MC	0	0	5	0	0	1326

Table 4.

Per-class performance metrics for subtype classification

Class	Precision	Recall	F1 score
CC	0.993	0.997	0.995
EC	0.992	0.992	0.992
HEALTHY	0.976	0.976	0.976
HGSC	0.994	0.992	0.993
LGSC	0.999	0.996	0.998
MC	0.993	0.996	0.994

Fig. 6.

Normalised confusion matrix

Compared to the approach by Kodipalli et al. [10], which achieved classification performance of 83% accuracy, 85% precision, and an F1 score of 81% for benign tumours and 79% accuracy, 85% precision, and 80% F1 score for malignant tumours, our method demonstrates substantially improved performance across a broader and more granular set of ovarian cancer subtypes. Kodipalli et al. employed a conditional generative adversarial network (cGAN) for image-to-image translation on a relatively small dataset, focusing solely on binary ovarian tumour classification. In contrast, our study leverages a larger and more diverse dataset that includes five distinct cancer subtypes and healthy tissue. Using a tile-based deep learning pipeline, our model achieved class-wise F1 scores ranging from 0.976 to 0.998 and an overall accuracy of 99.1%. These findings highlight both the scalability and the enhanced discriminative power of our approach over previous methods.

A major contributor to the model’s strong performance was the tiling strategy adopted in this study. Models trained without tiling not only performed similar to randomly guessing but also encountered significant computational limitations. In particular, whole slide images were too large to fit into GPU memory in their original resolution, necessitating aggressive downsampling. This process removed fine-grained histological details—such as nuclear morphology or subtle textural cues—that are critical for accurate subtype classification. In contrast, tiling allowed the model to process high-resolution patches that preserved these features, enabling it to focus on localised, biologically relevant patterns within each region. This decomposition simplified the classification task and enhanced the model’s ability to distinguish between subtypes. Additionally, tiling facilitated the detection of subtle tissue variations that might be overlooked when analysing entire slides. Overall, this proof-of-concept highlights tiling as not only essential for ovarian cancer subtype classification but also as a scalable and generalisable strategy for histopathological image analysis.

Conclusion

The EfficientNetB0 model achieved an overall test accuracy of $99.07\%\pm 0.7\%$ in ovarian cancer subtype classification. This suggests that the morphological and textural features of tumour regions are distinctive enough for the model to learn and differentiate effectively. Notably, the CC and HGSC subtypes were classified with near-perfect accuracy, while HEALTHY tissue and LGSC showed slightly more frequent misclassifications. A key contributor to these results was the tiling approach, which segmented images into smaller regions, allowing the model to focus on biologically meaningful features and improving classification precision. This strategy not only simplified the task but also enabled the capture of subtle tissue-level variations, all while overcoming GPU memory limitations.

Additionally, the absence of a detector in the loop made the classification problem easier to model, as it removed the need for additional object detection steps. This streamlined approach is beneficial when considering hardware resource limitations. Future work could focus on addressing misclassifications in the HEALTHY class and further refining performance through more diverse training data. The promising results underline the potential of this approach for improving ovarian cancer diagnosis and paving the way for personalised treatment strategies.

Author Contribution

SR wrote the manuscript and orchestrated the machine learning work. SR and RH implemented the data and machine learning pipelines. RH accelerated the data pipeline.

Declarations

Conflict of Interest

The authors declare no competing interests.