Deep Interactive Learning-based ovarian cancer segmentation of H&E-stained whole slide images to study morphological patterns of BRCA mutation

Abstract

Deep learning has been widely used to analyze digitized hematoxylin and eosin (H&E)-stained histopathology whole slide images. Automated cancer segmentation using deep learning can be used to diagnose malignancy and to find novel morphological patterns to predict molecular subtypes. To train pixel-wise cancer segmentation models, manual annotation from pathologists is generally a bottleneck due to its time-consuming nature. In this paper, we propose Deep Interactive Learning with a pretrained segmentation model from a different cancer type to reduce manual annotation time. Instead of annotating all pixels from cancer and non-cancer regions on giga-pixel whole slide images, an iterative process of annotating mislabeled regions from a segmentation model and training/finetuning the model with the additional annotation can reduce the time. Especially, employing a pretrained segmentation model can further reduce the time than starting annotation from scratch. We trained an accurate ovarian cancer segmentation model with a pretrained breast segmentation model by 3.5 hours of manual annotation which achieved intersection-over-union of 0.74, recall of 0.86, and precision of 0.84. With automatically extracted high-grade serous ovarian cancer patches, we attempted to train an additional classification deep learning model to predict BRCA mutation. The segmentation model and code have been released at https://github.com/MSKCC-Computational-Pathology/DMMN-ovary.

Graphical abstract

Highlights

• •A deep learning-based ovarian cancer segmentation model in pathology is developed
• •Only 3.5 hours of manual annotation were needed to train the segmentation model
• •BRCA mutation prediction is attempted based on H\&E-stained cancer morphology

1. Introduction

Deep learning, a subfield of machine learning, has shown an outstanding advancement in image analysis^1,2 by training models using large public datasets.3, 4, 5, 6 Deep learning models have been used to analyze and understand digitized histopathology whole slide images to support some tedious and error-prone tasks.7, 8, 9, 10 For example, a deep learning model was used to identify breast cancers to search micrometastases and reduce review time.¹¹ Similarly, another deep learning model was used as a screening tool for breast lumpectomy shaved margin assessment to save time for pathologists by excluding the majority of benign tissue samples.¹² In addition, deep learning models have been investigated to discover novel morphological patterns indicating molecular subtypes from histologic images.¹³ Correlating digitized pathologic images with molecular information has contributed to prognosis prediction and personalized medicine.¹⁴ Specifically, molecular features from lung cancer,¹⁵ colorectal cancer,^16,17 and breast cancer18, 19, 20 can be predicted by deep learning models from hematoxylin and eosin (H&E)-stained images.

All computational methods listed above either to diagnose cancers or to find biomarkers from cancer morphologies require accurate cancer segmentation of whole slide images. Unlike common cancers where public datasets with annotation are provided,21, 22, 23 training deep learning-based segmentation models for rare cancers would require a vast amount of manual annotation which is generally time-consuming. To overcome this challenge, we recently proposed Deep Interactive Learning (DIaL) to efficiently annotate osteosarcoma whole slide images to train a pixel-wise segmentation model.²⁴ During an initial annotation step, annotators partially annotate tissue regions from whole slide images. By iteratively training/finetuning a segmentation model and adding challenging patterns from mislabeled regions to the training set to improve the model, an osteosarcoma model was able to be trained by 7 hours of manual annotation.

In this paper, we develop an ovarian cancer segmentation model by 3.5 hours of manual annotation using DIaL. We hypothesize that we can utilize a pretrained triple-negative breast cancer (TNBC) segmentation model²⁵ to train a high-grade serous ovarian cancer (HGSOC) segmentation model because both TNBC and HGSOC show common high-grade carcinoma morphologies such as large pleomorphic nuclei and clumped chromatin. Transfer learning has been used when a data set is limited to train a model. The main challenge of training an ovarian cancer segmentation model is not the limited data set but the limited time for pathologists to annotate the data set. To reduce the annotation time from pathologists, our approach different from transfer learning finds essential regions to be annotated on a set of ovarian images. The main contribution of this work to train a HGSOC segmentation model is to start DIaL from a pretrained TNBC segmentation model²⁵ to reduce manual annotation time by avoiding the initial annotation step. Ovarian cancer accounts for approximately 2% of cancer cases in the United States but is the fifth leading cancer causing death among women and the leading cause of death by cancer of the female reproductive system.²⁶ HGSOC is the most common histologic subtype and accounts for 70-80% of deaths from all ovarian cancer.²⁷ Identifying BRCA1 or BRCA2 mutation status from HGSOC is important because family members of patients with germline mutations are at increased risk for breast and ovarian cancer and can benefit from early prevention strategies. In addition, it offers increased treatment options for making therapeutic decisions. Deleterious variants in the BRCA1 or BRCA2 genes are strong predictors of response to poly ADP-ribose polymerase (PARP) inhibitors such as olaparib, niraparib, and rucaparib.^28,29 Hence, the analysis of BRCA1/2 mutational status is crucial for individualized strategies for the management of patients with HGSOC. Identification of BRCA1/2 mutation is currently done by genetic tests but some patients may not be able to get the genetic tests due to its high cost and limited resources. Although these limitations have been overcome by many factors including reduced BRCA testing costs and next-generation sequencing, only 20% of eligible women have accessed genetic testing in the United States.³⁰ A cheaper approach to test BRCA1/2 mutation from H&E-stained slides is desired to examine wider range of ovarian cancer patients and to provide proper treatments to them. There has been an attempt to manually find morphological patterns of BRCA1/2 mutation.³¹ In this study, we conduct deep learning-based experiments to screen BRCA1/2 mutation from cancer regions on H&E-stained ovarian whole slide images automatically segmented by our model to potentially provide opportunities for more patients to examine their BRCA mutational statuses.

2. Material and methods

Fig. 1 shows the block diagram of our proposed method. Our method is composed of two steps: (1) ovarian cancer segmentation and (2) BRCA prediction. The goal of this study is to predict BRCA mutation from high-grade serous ovarian cancer, with a hypothesis that morphological patterns of the mutation would be shown on cancer regions. Therefore, we trained a deep learning model for automated segmentation of ovarian cancer from H&E-stained whole slide images to explore BRCA-related patterns on segmented cancer regions. Manual annotation process to train a deep learning-based segmentation model can be extremely time-consuming and tedious. To reduce annotation time, we used Deep Interactive Learning we previously proposed.²⁴ Our previous work had an initial annotation step to start manual annotation from scratch. One contribution of this work is that we used a breast pretrained segmentation model²⁵ as our initial model to start Deep Interactive Learning to further reduce annotation time by avoiding initial annotation. As a result, we were able to train our ovarian segmentation model with 3.5 hours of manual annotation. After the segmentation model was trained, we attempted to predict BRCA mutation based on cancer morphologies. We trained another model using ResNet-182 to generate patch-level scores indicating the probabilities of BRCA mutation. The patch-level scores were aggregated by averaging all patch-level scores to generate a slide-level score to classify an input slide image to either BRCA or non-BRCA. In this work, PyTorch³² was used for our implementation and an Nvidia Tesla V100 GPU was used for our experiments.

Fig. 1.

Block diagram of our proposed method to predict BRCA mutation from H&E-stained ovarian whole slide images. The first step was to segment ovarian cancer regions from whole slide images. To efficiently train the ovarian segmentation model, we used Deep Interactive Learning.²⁴ Since we started the process from a breast pretrained segmentation model, an annotator only spent 3.5 hours to annotate/correct whole slide images. After segmentation was done, cancer patches were processed by a BRCA classification model to generate patch-level scores indicating the probability of BRCA mutation. All patch-level scores were aggregated by averaging them to generate a slide-level prediction for BRCA mutation.

2.1. Data set

To segment ovarian cancer and to predict BRCA mutation based on tumor morphology, we collected 609 high-grade serous ovarian cancer cases at Memorial Sloan Kettering Cancer Center. The MSK-IMPACT whole slide images were digitized in 20× magnification by Aperio AT2 scanners. Approximately 20% of the cohort (119 images) have either BRCA1 or BRCA2 mutations and the other 80% of the cohort (490 images) have no BRCA mutation. We randomly split 60% of cases as a training set, 20% as a validation set, and the remaining 20% as a testing set, where the number of BRCA and non-BRCA images for training, validation, and testing are shown in Table 1.

Table 1.

The number of BRCA and non-BRCA whole slide images for our training, validation, and testing sets.

	BRCA	Non-BRCA	Total
Training Images	73	294	367
Validation Images	23	98	121
Testing Images	23	98	121
Total	119	490	609

2.2. Ovarian cancer segmentation

We hypothesize that morphological patterns caused by BRCA mutation would most likely be present in cancer regions on ovarian whole slide images. To train our BRCA prediction model at scale, we trained an ovarian cancer segmentation model to automatically extract cancer regions and avoid any time-consuming manual segmentation. In this work, we used Deep Multi-Magnification Network (DMMN) with multi-encoder, multi-decoder, and multi-concatenation²⁵ for ovarian cancer segmentation. DMMN generates a segmentation patch in size of 256 × 256 pixels in 20× based on various morphological features from a set of patches in size of 256 × 256 pixels from multiple magnifications in 20×, 10×, and 5×.

To train our DMMN model, manual annotation acquired from pathologists generally becomes a bottleneck. Hence, we adopted Deep Interactive Learning (DIaL)²⁴ to reduce time for manual annotation. As shown in Fig. 1, DIaL is composed of multiple iterations of segmentation, assessment, correction, and finetuning. In each iteration, the annotators assess segmentation predictions generated by the previous model and correct any mislabeled regions. The annotated patches are then included in a training set to train/finetune the segmentation model. These iterations during DIaL help the annotators to efficiently annotate challenging morphological patterns so the training set can contain heterogeneous patterns of classes. In our previous DIaL work, we started our initial annotation from scratch which took the majority of our annotation time. To further reduce annotation time, we utilized a pretrained segmentation model from another cancer type to skip the initial annotation step. In this work, we used a pretrained model to segment high-grade invasive ductal carcinoma from triple-negative breast cancer (TNBC) images²⁵ to train our model to segment high-grade serous ovarian carcinoma (HGSOC) because HGSOC and TNBC have shared morphological features such as large pleomorphic nuclei.

The pretrained model can segment 6 classes which are carcinoma, benign epithelium, stroma, necrosis, adipose tissue, and background. During DIaL iterations, we kept the model to segment 6 classes but we converted benign epithelium, stroma, necrosis, adipose tissue, and background to be non-cancer to have binary segmentation. The training set contained patches from both TNBC images and ovarian images. To optimize the model, we used stochastic gradient descent (SGD) with a weighted cross entropy loss function, where a weight for class c, w_c, was determined by $w_c=1-\frac{N_c}{N_t}$ where N_c was the number of annotated pixels for class c and N_t is the total number of annotated pixels. Random rotation, vertical and horizontal flips, and color jittering were used as data augmentation transformations.³³ During the first training, we trained the model from randomly initialized parameters with a learning rate of 5 × 10⁻⁵, a momentum of 0.99, and a weight decay of 10⁻⁴, where the same hyperparameters were used from our previous breast segmentation training.²⁵ After the first iteration, we finetuned the model from the previous parameters with a learning rate of 5 × 10⁻⁶. During training/finetuning iterations, we selected a model with the maximum intersection-over-union on a validation set as the final model of the iteration. The final segmentation model processes patches on tissue regions extracted by Otsu Algorithm.³⁴

2.3. BRCA prediction

With our hypothesis that BRCA morphological patterns would be shown on cancer regions, we trained a patch-level classification model predicting BRCA status from cancer patches. Cancer patches were extracted from cancer masks from the ovarian cancer segmentation model. A patch from a whole slide image was extracted if more than 50% of pixels in the patch were segmented as cancer. It was observed that the number of cancer patches from training images was imbalanced. If all cancer patches were used during training, morphological patterns on images with large cancer size would be more emphasized. Therefore, we set an upper limit, N^m, as the maximum number of patches from a training whole slide image. Specifically, if a whole slide image contained more than N^m patches, N^m training patches were randomly subsampled from the whole slide image. Otherwise, all patches from the whole slide image were included in the training set. In this work, we set N_BRCA^m = 5000 and N_nonBRCA^m = 1000 where N_BRCA^m was selected as the median value of the number of patches for BRCA cases, and N_nonBRCA^m was selected to balance the number of training patches between BRCA and non-BRCA classes. Note that we did not subsample patches from the validation and testing sets to produce slide-level predictions based on all cancer regions on whole slide images. The numbers of training, validation, and testing patches for BRCA and non-BRCA classes are shown in Table 2. We trained three models in three different magnifications, 20×, 10×, and 5×. We used ResNet-182 with patch size of 224 × 224 pixels. We used the same weighted cross entropy as a loss function, Adam³⁵ with learning rate of 10⁻⁵ as an optimizer, and used random horizontal and vertical flips, 90-degree rotations, and color jittering as data augmentation transformations.³³

Table 2.

The number of cancer patches for the training set, the validation set, and the testing set. Training patches between BRCA and non-BRCA are subsampled for balancing.

	BRCA	Non-BRCA	Total
Training Patches	247,059	252,166	499,225
Validation Patches	120,753	592,232	712,985
Testing Patches	142,983	624,231	767,214

Although our prediction model generates patch-level scores, p_patch, the final goal of this work is to classify BRCA mutation in slide-level. Hence, we validated and tested our models in slide-level by aggregating patch-level scores. In this work, a slide-level score, p_slide, was calculated by averaging the N patch scores in an input whole slide image:

$$p_{\mathit{slide}}=\frac{\sum p_{\mathit{patch}}}{N}$$ (1)

We used area-under-curves (AUCs) as our evaluation metric.

2.4. Ethics declarations

This study was approved by the Institutional Review Board at Memorial Sloan Kettering Cancer Center (Protocol #21-473).

3. Results

3.1. Training an ovarian cancer segmentation model with Deep Interactive Learning

We iteratively trained our ovarian cancer segmentation model using Deep Interactive Learning (DIaL) where annotation was done by an in-house slide viewer.³⁶ We randomly selected 60 whole slide images from our training set where 20 images with BRCA1 status, 20 images with BRCA2 status, and 20 images with no BRCA status were selected to make sure our ovarian cancer segmentation model can successfully segment all molecular subtypes. After segmenting all 60 images using the pretrained breast model, denoted as $\mathcal{M}$₀, we observed papillary pattern for carcinoma and ovarian stroma were mislabeled because these patterns were not presented on triple-negative breast cancer images. The annotator corrected mislabeled regions on 14 whole slide images to train the first model denoted as $\mathcal{M}$₁, which took approximately 1 hour. Note that papillary pattern is combined within carcinoma class and ovarian stroma is combined within stroma class without introducing new class. During the second iteration, we observed that carcinoma and stroma were segmented correctly but some challenging patterns such as fat necrosis and fat cells in lymph node were mislabeled by $\mathcal{M}$₁. The annotator looked for those challenging patterns in detail and annotated 11 whole slide images (3 images overlapping with the first correction step) to train the second model denoted as $\mathcal{M}$₂, took approximately 2 hours. During the third iteration, we observed that some markers were mislabeled as carcinoma by $\mathcal{M}$₂, so the annotator corrected the other 3 whole slide images in 30 minutes to train the third model denoted as $\mathcal{M}$₃. After finetuning the model, we observed $\mathcal{M}$₃ can successfully segment cancers on the training set so we completed the training stage. In total, we annotated 25 ovarian whole slide images and spent 3.5 hours. As a comparison, exhaustively annotating cancer regions on one ovarian whole slide image without DIaL took 1.5 hours, indicating one would be able to annotate only 2–3 training whole slide images within the same 3.5 hours. Fig. 2 shows mislabeled regions by a segmentation model and corrected regions using DIaL.

Fig. 2.

Deep Interactive Learning for efficient annotation to train an ovarian cancer segmentation model. Instead of annotating all regions on ovarian whole slide images, the annotator can only annotate a subset of regions to train/finetune the ovarian cancer segmentation model. (a–c) The first iteration of correction of mislabeled cancer regions from $\mathcal{M}$₀. (d–f) The first iteration of correction of mislabeled stroma regions from $\mathcal{M}$₀. (g–i) The second iteration of correction of mislabeled fat necrosis regions from $\mathcal{M}$₁. Cancers are highlighted in red, stroma in green, and adipose tissue in orange.

3.2. Ovarian cancer segmentation evaluation

To quantitatively analyze ovarian cancer segmentation models, another pathologist who was not involved in training manually generated groundtruth for 14 whole slide images randomly selected from the testing set. Intersection-over-union (IOU), recall, and precision are used to evaluate segmentation models, where they are defined as:

$$\mathit{IOU}=\frac{N_{\mathit{TP}}}{N_{\mathit{TP}}+N_{\mathit{FN}}+N_{\mathit{FP}}}$$ (2)

$$\mathit{Recall}=\frac{N_{\mathit{TP}}}{N_{\mathit{TP}}+N_{\mathit{FN}}}$$ (3)

$$\mathit{Precision}=\frac{N_{\mathit{TP}}}{N_{\mathit{TP}}+N_{\mathit{FP}}}$$ (4)

where N_TP, N_FN, and N_FP are the number of true-positive pixels, the number of false-negative pixels, and the number of false-positive pixels, respectively. Table 3 shows IOU, recall, and precision values for $\mathcal{M}$₀, $\mathcal{M}$₁, $\mathcal{M}$₂, and $\mathcal{M}$₃ where the final model achieved IOU of 0.74, recall of 0.86, and precision of 0.84. A high precision value and a low recall value from $\mathcal{M}$₀ indicate that the initial model trained by triple-negative breast cancer was not able to segment all high-grade serous ovarian cancer. After the first iteration by adding papillary patterns for carcinoma, both the IOU value and the recall value were significantly improved. The second and third iterations had minor updates for correction so the IOU value, the recall value, and the precision value were not significantly improved. We were able to achieve the highest recall value from the final model indicating the majority of high-grade serous ovarian cancer regions were successfully segmented and heterogeneous morphological patterns would be used to train the BRCA classification model. Fig. 3 and Supplementary Fig. S1 and S2 show that our final model can successfully segment ovarian cancers present on three testing whole slide images. We observed the final model generates false negatives and false positives, shown in Supplementary Fig. S3 and S4. Specifically, false negatives we observed were caused by cautery artifact or poor staining. False positives were caused by smooth muscles on fallopian tube, colon epithelium, or blood vessels which were underrepresented in the training data. By including normal tissue samples from metastasized cases from other organ types in the training set to further finetune our segmentation model, we expect to reduce false positives. The segmentation model and code have been released at https://github.com/MSKCC-Computational-Pathology/DMMN-ovary.

Table 3.

Intersection-over-union (IOU), recall, and precision for the initial model ($\mathcal{M}$₀), the first model ($\mathcal{M}$₁), the second model ($\mathcal{M}$₂), and the final model ($\mathcal{M}$₃). Note that the initial model is the pretrained breast model.²⁵ The highest IOU, recall, and precision values are highlighted in bold.

	IOU	Recall	Precision
$\mathcal{M}$0	0.65	0.68	0.93
$\mathcal{M}$1	0.74	0.84	0.86
$\mathcal{M}$2	0.72	0.81	0.87
$\mathcal{M}$3	0.74	0.86	0.84

Fig. 3.

Ovarian image, its groundtruth, and its segmentation. (a–c) show the entire whole slide image and (d–f) show a zoom-in image. Cancers are highlighted in red. White regions in (b,e) and gray regions in (c,f) are non-cancer.

3.3. BRCA prediction

Based on cancer segmentation, we trained three BRCA classification models in 20×, 10×, and 5×, denoted as $\mathcal{M}$_20×, $\mathcal{M}$_10×, and $\mathcal{M}$_5×, respectively. During training, the model with the highest area-under-curves (AUCs) on the validation set was selected as the final model. Table 4 shows AUCs on the validation set and the testing set using the three models in various magnifications where the AUCs were ranging between 0.49 and 0.67 on the validation set and between 0.40 and 0.43 on the testing set.

Table 4.

Area-under-curves (AUCs) of three classification models on the validation set and the testing set.

	$\mathcal{M}$20×	$\mathcal{M}$10×	$\mathcal{M}$5×
Validation AUC	0.49	0.65	0.67
Testing AUC	0.40	0.42	0.43

4. Discussion

In this paper, we described Deep Interactive Learning (DIaL) which helps annotators to reduce their annotation time to train deep learning-based pixel-wise segmentation models. Our ovarian cancer segmentation model was able to accurately segment cancer regions presented in H&E-stained whole slide images with intersection-over-union of 0.74, recall of 0.86, and precision of 0.84.

Cancer segmentation of histologic whole slide images is used to accurately diagnose malignant tissue. For example, a patch-wise model is designed to identify invasive carcinoma in breast whole slide images.³⁷ Multiple techniques for automated breast cancer metastasis detection in lymph nodes^38,39 has been developed through challenges such as CAMELYON16²¹ and CAMELYON17.²² For more accurate segmentation, pixel-wise semantic segmentation models such as Fully Convolutional Network (FCN),⁴⁰ SegNet,⁴¹ and U-Net⁴² have been utilized on whole slide images.43, 44, 45 One limitation of these semantic segmentation models is that their input is a patch from a single magnification, where pathologists generally review tissue samples via a microscope in multiple magnifications for cancer diagnosis. To overcome this challenge in pathology, Deep Multi-Magnification Network (DMMN) utilizing a set of patches from multiple magnifications has been proposed.²⁵ In DMMN, patches from 20×, 10×, and 5× magnifications in a multi-encoder, multi-decoder, multi-concatenation architecture are fully utilized to fuse morphological features from both low magnification and high magnification. The proposed segmentation network outperformed other single-magnification-based networks.

Cancer segmentation is a critical process not only for diagnosis but also for downstream tasks such as molecular subtyping from H&E-stained whole slide images. Molecular status is currently detected by genetic tests but the genetic tests are generally costly and may not be available for all patients. Deep learning models as screening tools can help patients to get proper treatment from cheap H&E stains.⁴⁶ Six mutations were predicted from patches classified as lung adenocarcinoma.¹⁵ Microsatellite instability (MSI) status was predicted from patches classified as gastrointestinal cancer.¹⁶ More molecular pathways and mutations in colorectal cancer were predicted on tumor regions.¹⁷ Furthermore, molecular alterations from 14 tumor types were predicted from tumor patches.⁴⁷ BRCA mutation¹⁸ and HER2 status²⁰ in breast cancer were predicted from tumor patches on H&E-stained images. To train deep learning models to determine molecular subtypes from H&E-stained images, manual annotation of tumor regions on whole slide images was required.^18,20,47 To avoid manual annotation of tumor regions for downstream predictions, automated segmentation would be desired.

Alternatively, weakly-supervised learning has been proposed to avoid manual annotation of cancer regions to train cancer segmentation models. Weakly-supervised learning provides approaches to train classification models by weak labels, such as case-level labels instead of pixel-level labels.^48,49 Weakly-supervised learning can be a promising solution for common cancers because it require a large training set representing one whole slide image as one data point. For example, a weakly-supervised method to predict estrogen receptor status in breast cancer was trained by 2,728 cases.⁵⁰ However, for relatively rare cancers such as high-grade serous ovarian cancer (HGSOC) where the number of cases is limited, weakly-supervised learning can be challenging to detect morphological patterns representing molecular mutations on whole slide images. In our study, the total number of cases of HGSOC was 609 where only 20% has BRCA status (119 cases). Therefore, instead of weakly-supervised learning, supervised mutation predictions from cancer segmentation would be more proper especially for rare cancers with limited number of cases.

For automated cancer segmentation, reducing manual annotation time on digitized histopathology images to train segmentation models has been a practical challenge. Deep learning-based approaches require large quantities of training data with annotations, but pixel-wise annotation for a segmentation model is extremely time-consuming and especially difficult for pathologists with their busy clinical duty. To reduce annotation burden, approaches to train cell segmentation models with a few scribbles were suggested.^51,52 Human-Augmenting Labeling System (HALS) introduced an active learner selecting a subset of training patches to reduce annotation burden for cell annotation.⁵³ To segment tissue subtypes, an iterative tool, known as Quick Annotator, was developed to speed up the annotation time within patches extracted from whole slide images.⁵⁴ Patch-level annotation may limit tissue subtypes’ field-of-view, potentially causing poor segmentation.^25,55 Deep Interactive Learning (DIaL)²⁴ was proposed to efficiently label multiple tissue subtypes in whole slide image-level to reduce time for manual annotation but to have accurate segmentation. After an initial segmentation training based on initial annotations, mislabeled regions are corrected by annotators and included in the training set to finetune the segmentation model. As challenging or rare patterns are added during correction, the model can improve its segmentation performance iteratively. Within 7 hours of manual annotation, the osteosarcoma segmentation model achieved an error rate between its multi-class segmentation predictions and pathologists’ manual assessment within an inter-observer variation rate.²⁴ In this paper, we further reduced the annotation time by starting from a pretrained segmentation model from a different cancer type to skip the initial annotation step. In our case, we used a triple-negative breast cancer (TNBC) segmentation model²⁵ as our initial model and the annotator spent only 3.5 hours to train an accurate HGSOC segmentation model. We considered to use a TNBC segmentation model to train HGSOC segmentation model because both TNBC and HGSOC share high-grade carcinoma morphologic features. It is remained as a future step to generalize our approach by quantitatively comparing cellular representations between a training set for a pretrained model and a testing set.

We desired to predict BRCA mutation status from cancer regions from H&E-stained ovarian whole slide images. BRCA mutation status can determine patients’ future treatment but it is currently detected by expensive genetic examinations. A deep learning-based tool to screen potential BRCA cases for genetic examinations from cheap and common H&E staining would expect to enhance patients’ treatment and outcome. Based on our experiments, we were not able to discover morphological patterns on ovarian cancer indicating BRCA mutation. Several future steps are proposed: (1) We had a hypothesis that the BRCA-related morphological patterns may be shown on cancer regions. In the future, we may want to expand our hypothesis to non-cancer regions.⁵⁶ For example, one could train a model from tumor-stroma regions. (2) We had a hypothesis that the BRCA-related morphological patterns may be shown globally. Therefore, we included all cancer patches to the training set with case-level labels. If the BRCA patterns are shown locally (i.e., some cancer regions may not contain the BRCA pattern although its case is labeled as BRCA), then weakly-supervised learning approaches^48,49 may be a better option by increasing the number of cases. (3) Lastly, cancer morphologies with BRCA mutation could be heterogeneous because their mutational spectrum is highly heterogeneous.⁵⁷ A self-supervised clustering technique could be used to discover multiple morphological patterns of BRCA mutation by increasing the number of BRCA-mutated cases.

In conclusion, we developed an accurate deep learning-based pixel-wise cancer segmentation model for ovarian cancer. Especially, by Deep Interactive Learning with a pretrained model from breast cancer, we were able to reduce manual annotation time for training. Although our study had suboptimal performance on predicting BRCA mutation based on morphological patterns, we are confident that our ovarian segmentation model can be used to discover other mutation-related patterns from H&E-stained images for screening tools to determine treatment and to enhance patient care.

Data availability

The segmentation model and code are publicly available at https://github.com/MSKCC-Computational-Pathology/DMMN-ovary. The data set used and/or analyzed during the current study is available on reasonable request.