Assessing the impact of deep‐learning assistance on the histopathological diagnosis of serous tubal intraepithelial carcinoma (STIC) in fallopian tubes

Abstract

In recent years, it has become clear that artificial intelligence (AI) models can achieve high accuracy in specific pathology‐related tasks. An example is our deep‐learning model, designed to automatically detect serous tubal intraepithelial carcinoma (STIC), the precursor lesion to high‐grade serous ovarian carcinoma, found in the fallopian tube. However, the standalone performance of a model is insufficient to determine its value in the diagnostic setting. To evaluate the impact of the use of this model on pathologists' performance, we set up a fully crossed multireader, multicase study, in which 26 participants, from 11 countries, reviewed 100 digitalized H&E‐stained slides of fallopian tubes (30 cases/70 controls) with and without AI assistance, with a washout period between the sessions. We evaluated the effect of the deep‐learning model on accuracy, slide review time and (subjectively perceived) diagnostic certainty, using mixed‐models analysis. With AI assistance, we found a significant increase in accuracy (p < 0.01) whereby the average sensitivity increased from 82% to 93%. Further, there was a significant 44 s (32%) reduction in slide review time (p < 0.01). The level of certainty that the participants felt versus their own assessment also significantly increased, by 0.24 on a 10‐point scale (p < 0.01). In conclusion, we found that, in a diverse group of pathologists and pathology residents, AI support resulted in a significant improvement in the accuracy of STIC diagnosis and was coupled with a substantial reduction in slide review time. This model has the potential to provide meaningful support to pathologists in the diagnosis of STIC, ultimately streamlining and optimizing the overall diagnostic process.

Introduction

The field of histopathology is digitalizing. Pathology images can be scanned to high‐resolution images [whole slide images (WSIs)], which pathologists can assess on a computer screen, instead of using a traditional microscope [1, 2, 3, 4]. With the digitalization of pathology, new opportunities to aid pathologists in their daily practice, such as artificial intelligence (AI), and more specifically deep‐learning (DL) algorithms, emerge [5, 6]. Such algorithms have already proven successful in detection and classification tasks, reaching accuracy levels that are similar to (subspecialized) pathologists [7, 8]. In recent years, many AI models have been developed, mostly addressing major subareas in pathology, e.g. breast and prostate cancers [9, 10]. However, AI algorithms also offer an interesting opportunity to improve diagnostics for rare diseases, which many pathologists may encounter only infrequently and for which (sub)specialized expertise might be lacking. One such example is the precursor lesion to high‐grade serous ovarian carcinoma (HGSC), known as serous tubal intraepithelial carcinoma (STIC).

STIC is a noninvasive lesion, found in the fallopian tube, showing morphologic alterations similar to HGSC [11]. The clinical impact of a diagnosis of STIC is high, holding important prognostic information for individual patients and therapeutic consequences in studies on alternative risk‐reducing strategies, where a diagnosis of isolated STIC could lead to an oophorectomy [12, 13]. However, STIC can be a challenging diagnosis for pathologists and the interobserver agreement is moderate at best, when based on H&E‐stained slides [14, 15, 16]. A diagnosis of STIC is therefore often supported by additional immunohistochemical stains for p53 and Ki‐67. These may also help differentiate STIC from serous tubal intraepithelial lesion (STIL), a lesion that falls short of the criteria for STIC, but potentially also holds a link to HGSC [14, 17, 18]. Next to the classification, detecting STIC is likewise challenging. The incidence of isolated STIC (not in combination with concomitant HGSC) is low and ranges from 0.1% in populations with normal risk for ovarian carcinoma, up to 3% in risk‐reducing salpingoophorectomy specimens from women at an increased risk of HGSC, such as BRCA1/2 pathogenic variant carriers [19, 20]. This means that the average pathologist will have relatively little exposure to these lesions. Moreover, isolated cases of STIC and STIL are often very small and are easily overlooked [21]. A special grossing protocol, called the sectioning and extensively examining the fimbriated end (SEE‐FIM) protocol was developed to maximize the likelihood of detection of STIC, but has resulted in a marked increase in the number of slides that a pathologist has to screen [22].

To assist pathologists in detecting potential STIC/STIL, we previously developed a DL algorithm, which was proven capable of accurately detecting regions of potential STIC/STIL in a fully automated fashion [23]. The aim of this study is to evaluate the impact of the use of this DL model on pathologists' performance in STIC detection, focusing on the effect on accuracy and slide review time of (gynecologic) pathologists and pathology residents.

Materials and methods

Case enrollment

One hundred specimens were selected from a previously established study cohort that was compiled at the time that the DL model was developed [23]. These specimens were previously used for validation and were not used for model training. The set contains 21 cases of STIC, 9 cases of STIL, and 70 controls. From each specimen, a single slide was selected. The cases originate from three different data sources, namely Radboud University Medical Center, the Netherlands (n = 7), the Dutch Nationwide Network of Histopathology and Cytopathology database (PALGA) (n = 14), and Johns Hopkins University, MD, USA (n = 9). Controls were also acquired from three different data sources: Radboud University Medical Center (Radboudumc), Canisius Wilhelmina Hospital Nijmegen (CWZ), and Rijnstate Hospital Arnhem, all in the Netherlands. This project was reviewed and approved by the research ethics committee at the Radboudumc (number: 2019‐5879). Data transfer agreements were established with all involved parties, and ethical approval for the research was obtained in accordance with the local regulations (PALGA, ref. LZV_55‐A1; Johns Hopkins University, USA, Ref TTO RUMC A22‐0963; IRB: 00203161).

Reference standard

A reference standard for the previously established cohort, containing 249 slides diagnosed with STIC or STIL and 247 control slides had been set, including the 21 cases from Radboudumc and PALGA used in this study. Still images of regions of interest from these slides, identified by a pathology resident, under supervision of a gynecologic pathologist, were shown to an international panel of 15 experienced gynecologic pathologists, whereby each image was reviewed by 5 pathologists from this panel. These pathologists, blinded to the original diagnoses, classified the images into seven categories like ‘normal’, ‘STIC’, ‘STIL’, etc. As various diagnostic algorithms for STIC exist, no specific instructions regarding morphological criteria or immunohistochemistry (IHC) were given to this panel [14, 17, 18]. Whenever IHC for p53 and Ki‐67 were available, these were provided to the panel, to assist in classifying the lesion. IHC was not further used in training the AI model. The expert panel reached a median kappa value of 0.53 when asked to subclassify the lesions. When only asked to make distinction between aberrant epithelium (STIC/STIL) versus normal epithelium, they reached a median kappa value of 0.86, corresponding to a strong level of agreement [23, 24]. A definitive label of STIC/STIL was assigned when at least three pathologists agreed on the diagnosis. Based on these labels, annotations were made by the pathology resident on a cellular level, and checked by the gynecologic pathologist, who was not a member of the expert panel. The nine cases from Johns Hopkins University, which in the previous study were used in the external test set for the algorithm, as well as the 70 controls were reviewed at Radboudumc by a subspecialized gynecological pathologist (MS) and a pathology resident (JMAB). Pathologists involved in setting the reference standard were excluded from participating in this reader study. A more detailed description on setting the reference standard, including kappa values of the panel and visual examples of the annotations process are available in the article describing the models development and validation [23].

Readers

A total of 26 pathologists and pathology residents, hereafter referred to as ‘readers’ participated. None of them had been involved in setting the reference standard. The readers came from 11 countries, and self‐reported to be working as pathology resident (7), (general) pathologist in a nonacademic hospital (2), gynecologic pathologist in a nonacademic hospital (1), (general) pathologist in an academic hospital (3), or gynecologic pathologist in an academic hospital (13). Their years of work experience (including residency years) ranged between 3 and 38, with a median of 10 years. Levels of experience with digital pathology varied from none (n = 3), in research setting only (14), to working in a laboratory with a partially (4) or fully (5) digitalized clinical workflow. An overview of the readers' background information is provided in supplementary material, Section S1.

DL algorithm

We previously developed a DL algorithm which is capable of automatically detecting regions of aberrant epithelium (STIC/STIL) in digitalized H&E‐stained WSIs of fallopian tubes [23]. In brief, the model consists of a U‐Net with a resnet50 backbone and was trained using 118 cases of STIC/STIL and 51 controls. The model was evaluated on two independent test sets, containing 327 and 186 slides respectively. The model displayed a robust performance, whereby receiver operating characteristic curve analysis achieved an area under the curve of 0.98 (95% CI: 0.96–0.99) on test set one, and 0.95 (95% CI: 0.90–0.99) on test set two.

Study design

This study was set up with a fully crossed multireader multicase design, as illustrated in Figure 1 [25]. All 26 readers took part in two sessions, reviewing the same 100 slides twice. In between these two sessions there was a minimum washout period of 4 weeks. The set of images was divided into blocks of 10 or 15 images. Each block contained a similar distribution of cases versus controls, to mitigate bias for potential differences in performance at the beginning versus the end of a session. Readers were randomly split into two groups beginning with either order 1 or order 2. In either order, the sequence of slides was identical. The sole difference between orders 1 and 2 was whether the blocks of 10–15 images were reviewed with or without AI assistance. Readers could work on the study at their own discretion and did not have to complete a session within a single sitting. In order to let readers familiarize themselves with reviewing digital WSIs in the viewer, two test slides were provided at the beginning of each session, which were not part of data analysis. At the end, all 26 readers had reviewed all 100 slides in two modalities, once with AI assistance (using a visual representation of the DL model output; assisted read), and once without AI assistance (unassisted read).

Figure 1.

Schematic display of the study design. Readers were randomly assigned to an order. Images in both orders were the same and were displayed in the same sequence but were reviewed either with or without AI assistance. Each square represents a set of images, with the number indicating the number of images in that set. The color indicates whether AI assistance for that set was available to the reader.

Research platform

The study was performed using the online CastorEDC study platform (https://www.castoredc.com). Readers received separate invitation links to the different sessions. Readers could access the WSI of H&E‐stained slides by clicking a link in the study platform, which opened the slide in the Pathomation WSI viewer (www.pathomation.com), in a separate tab. If AI assistance was available, this was visualized as a black bounding box, surrounding the area(s) the algorithm had detected as aberrant (Figure 2). Slides could contain multiple bounding boxes. A number, indicating the number of bounding boxes, was displayed at the top of the digitalized image. If the algorithm had not detected any regions of potential aberrant epithelium, the number would be displayed as 0 (zero). After reviewing the WSI, readers were asked to indicate whether the slide contained aberrant epithelium (STIC/STIL), based on assessment of the H&E slide. In addition, readers indicated how certain they felt in their assessment on a 1–10 scale, with 1 indicating a low level of certainty and 10 indicating a high level of certainty. When readers reported that a WSI contained a (potential) STIC/STIL, they received a follow up question as to whether they would request additional IHC: often a pathologist will want to confirm the final diagnosis by IHC. Readers could specify if the request for IHC was for diagnostic confirmation of STIC, or for assistance in classifying the aberrant epithelium identified.

Figure 2.

AI assistance was visualized as a black bounding box, surrounding the area(s) the algorithm had detected as aberrant (A1, B1, C1, and D1). The AI model output can also be visualized as a color‐coded heatmap (A2, B2, C2, and D2), whereby absence of color indicates that the model predicts no presence of STIC/STIL, green indicates a low probability and red indicates a high probability of STIC/STIL being present. Readers of the study did not see these heatmaps. They are presented here to show what areas the AI models base its predictions on. In the reference standard, lesions A and B had been diagnosed as STIC; lesions C and D had been diagnosed as STIL. During the study, readers only had to indicate whether they thought STIC or STIL was present. They did not have to make a classifying diagnosis.

Image review timing

Readers were instructed to assess slides at a self‐controlled pace. The review time for each slide was obtained by calculating the time difference between the moment that the WSI was opened in the viewer, to the moment that the first question was answered. This time difference was calculated by using the application programming interface of the CastorEDC research platform. Readers were instructed not to take breaks while reviewing a slide.

Statistical analysis

Statistical analyses for ‘accuracy’, ‘slide review time’, and ‘certainty’ were performed using mixed‐effects models. All models were generated using the lme4 package in R [26, 27]. In these models, readers and images were treated as random effects, whereby images were nested within readers. The assistance modality (with or without AI assistance) and the session (having started with order 1 or 2), were treated as fixed effects, with an interaction. To evaluate the readers ‘accuracy,’ a binominal generalized linear mixed model was generated, using the glmer function. The odds ratio was obtained by taking the exponent of the estimated effect. To evaluate the readers ‘slide review time’ and ‘level of certainty’ a linear mixed model was generated, using the lmer function. Fleiss' kappa values for the evaluation of interobserver agreement were calculated using IBM SPSS statistics version 27. The sensitivity and specificity of each reader was calculated for both the assisted and unassisted reads, for descriptive purposes.

Results

Performance and accuracy

An increase in sensitivity and specificity was observed when using AI assistance, whereby the median sensitivity for assisted reads was 93%, compared to 82% for the unassisted reads (Figure 3A). Also, the interquartile range reduced when AI assistance was used. This is also reflected in the increase in interobserver agreement, whereby the Fleiss' kappa (k) increased from (k) = 0.506 (95% CI: 0.495–0.517) for unassisted reads, corresponding with a moderate reproducibility, to (k) = 0.725 (95% CI: 0.714–0.736) for assisted reads, corresponding to good reproducibility [24]. Out of the 26 readers, 22 had an increased sensitivity with the assisted reads. The remaining four readers retained the same sensitivity (supplementary material, Section S2). Specificity increased from 92% to 97% as a result of AI assistance. The three cases that reached the greatest increase in accurate detection are displayed in Figure 4. A significantly higher likelihood to accurately detect STIC/STIL with the assisted reads compared to the unassisted reads was found, with an odds ratio of 6.49 (95% CI: 3.39–12.40) (p < 0.01).

Figure 3.

Results from descriptive statistics. (A) Boxplot displaying the distribution of readers' average sensitivity and specificity for both assisted and unassisted reads. (B) Boxplot displaying the distribution of readers' average slide review time for assisted and unassisted reads. (C) Boxplot displaying the distribution of readers' sense of certainty (on a 1–10 scale) for both assisted and unassisted reads. (D) Bar chart displaying the requested immunohistochemistry in absolute numbers, under assisted and unassisted reads.

Figure 4.

(A1) ×40 magnification of an H&E‐stained slide, showing STIC with overexpression in the corresponding p53 stain (A2) and increased proliferative activity shown with Ki‐67 (A3). (B1) ×40 magnification of an H&E‐stained slide, showing STIL, with a (dubious) null pattern in the p53 stain (B2) and increased proliferative activity compared to surrounding epithelium, but <10% (B3). (C1) ×40 magnification of an H&E‐stained slide, showing STIL, with overexpression of p53 (C2) and a slight increase in proliferative activity with Ki‐67 positivity <10% (C3).

Slide review time

The slide review time decreased significantly, with an estimated effect of 44 s less time spent in assisted reads compared to the unassisted reads (95% CI: 37–51) (p < 0.01). This corresponds to an average time reduction of 32% for the assisted reads. The time reduction was observed both for cases (reduction of 49 s; 95% CI: 37–61; p < 0.01) and for controls (reduction of 41 s; 95% CI: 33–49; p < 0.01). All 26 readers were faster with the assisted reads. Figure 3B visualizes the slide review times for assisted and unassisted reads, whereby the median review time decreased from 156 s for the unassisted reads to 114 s for the assisted reads. Their individual average slide review times are listed in supplementary material, Section S3.

Certainty score

A significant increase in readers' subjective certainty score was found, with an estimated effect of 0.25 increase for the assisted reads compared to the unassisted reads (95% CI: 0.13–0.36; p < 0.01). This effect was primarily observed among the controls (increase of 0.35; 95% CI: 0.22–0.46; p < 0.01) but was not significant in the cases (increase of 0.03; 95% CI: −0.22 to 0.28; p = 0.8). Out of the 26 readers, 21 had an increased sense of certainty with AI assistance. Figure 3C visualizes the individual readers' certainty scores for assisted and unassisted reads, whereby the median certainty score increased from 7.94 for unassisted reads to 8.60 for the assisted reads. The readers' individual average certainty scores are listed in supplementary material, Section S4.

IHC requests

In the unassisted reads, there were 775 instances where readers thought that STIC/STIL was present. Among these there were 683 requests for additional IHC. In the assisted reads, there were 791 potential STIC/STIL observed, resulting in 669 requests for IHC. An overview of the responses is depicted in Figure 3D.

Subgroup analysis

Subgroup analysis was based on work environment (residents; nongynecologic pathologists and gynecologic pathologists); years of work experience [≤10 years (median) versus >10 years] and level of experience with digital pathology [readers with experience limited to research settings versus readers working in a laboratory with an (at least) partially digitalized clinical workflow]. Analysis for accuracy using the generalized linear mixed‐model was unsuccessful due to the complicated model in relation to the limited size of several subgroups, which led to overfitting. Analysis from the mixed models for ‘slide review time’ and ‘level of certainty’ was successful. However, due to the limited sample size within some of these subgroups, any conclusions drawn from these analyses should be interpreted with caution. The small sample size limits the reliability of the findings. These results, along with descriptive analysis of the sensitivity and specificity, are presented in supplementary material, Section S5.

Readers' experience

A brief questionnaire was held at the end of each session in which readers were asked about their experiences using AI assistance. The results show that, in general, most readers had a positive attitude toward the use of AI assistance and would be willing to integrate DL assistance for STIC/STIL detection in their clinical workflow (Figure 5).

Figure 5.

Results of the user satisfaction survey. Readers were asked to rate their level of agreement on a 5‐point scale, ranging from ‘strongly disagree’ to ‘strongly agree’. Numbers in the colored segments reflect the percentage of votes.

Discussion

In this study, we assess the effect of AI assistance on pathologists' and pathology residents' performance when screening for STIC/STIL in H&E‐stained slides of fallopian tube specimens. We found a significant increase in the accuracy of detecting STIC, combined with a significant decrease in slide review time and a rise in the level of certainty experienced.

A total of 22 out of the 26 readers achieved a higher sensitivity with AI assistance. Furthermore, the overall spread of the individual sensitivity and specificity scores decreased, as demonstrated by the reduced interquartile ranges when AI assistance was used. This increase in interobserver agreement is also reflected in the increase in kappa value (k), which went from (k) = 0.506 for the unassisted reads, corresponding to moderate reproducibility, to (k) = 0.725 for the assisted reads, corresponding to good reproducibility [24]. The increased accuracy under AI assistance appeared to persist in the subgroups, but these subgroups were too small to support the mixed‐effects model. The potential for AI models to achieve high accuracy in specific pathology‐related tasks has been previously reported, and multiple examples where DL models achieve high levels of accuracy, on par with pathologists, are available [7, 9, 10, 28]. However, the ‘stand‐alone’ performance of AI holds little value for assessing the practical diagnostic value of such a model, as such algorithms are not intended to replace pathologists [2, 29]. It is the constructive interaction between pathologist and DL assistance which has exciting potential to further strengthen diagnostic accuracy, supporting the pathologists and enabling them to perform at the highest possible level. Studies such as the current one, in which a panel of pathologists using DL assistance is compared to a baseline, are a critical next step in translating research models to clinical use [7].

The increased sensitivity seems indicative of the AI highlighting regions to the readers that they may have overlooked in the unassisted setting. In many applications of the use of AI, such increased sensitivity comes at the cost of decreased specificity: a sensitive algorithm typically also produces false positives that may lead to cases being overcalled. Interestingly, in the current study, improved sensitivity was paired with an increase in specificity. Possibly, the quality of the AI was such that in a number of hard (false positive) regions where the pathologist was in doubt, AI tipped the decision in the correct direction.

In addition, the results showed a significant reduction in slide review time. The mixed models calculated AI assisted reads to be 44 s faster, consistent with a time reduction of 32%. This effect was visible in both cases and controls, where the estimated time reduction was 41 s. STIC is a relatively rare entity in isolation [19, 20]. Thus, pathologists spend the majority of their time reviewing STIC/STIL negative slides. If a time reduction is to be clinically meaningful, this should be achieved in the negative slides. With this significant time reduction among controls, an even more rigorous assessment of fallopian tube specimens, i.e. by more widespread use of the SEE‐FIM grossing protocol, or examination of tissue blocks on multiple levels, might become feasible in a study setting, to analyze its potential added value [22, 30, 31]. Though time reduction is welcomed, the potential risk for over‐reliance on an algorithm has been debated [32]. Therefore, it is important for pathologists to acquire skills and knowledge about how AI support works. They must be able to grasp both the potential advantages and limitations of AI assistance, as well as have access to tools that aid in understanding or visualizing the AI model's predictions. One example of such a tool is the use of heatmaps to visualize AI model output, as shown in Figure 2. Building these skills is essential for pathologists to better comprehend the role of AI in their work and to ensure the safe and responsible integration of these technologies into daily practice. It is important to keep in mind that time reduction is not in itself the primary goal, and is only secondary to (improved) accuracy.

Finally, a modest increase in certainty was observed for the assisted reads. When analyzing cases and controls separately, we see that this increase in certainty mostly results from assessment of the controls (0.35 increase; p < 0.01) rather than from cases (0.03 increase; p = 0.8). Potentially, the idea of an additional check, making sure that a lesion has not been missed, can provide that additional sense of certainty. It must be noted that any study on the impact of AI on pathologists' diagnostics suffers from the fact that, today, AI models are hardly ever used in pathology, meaning that the study participants have no experience in the use of such diagnostic support. It is likely that once a pathologist has more experience with AI and better understands the strengths and weaknesses, the benefit of using AI will only increase.

This study has several limitations. The dataset was enriched with cases of STIC/STIL, as mimicking the normal incidence of isolated STIC would make a study onerous and infeasible. In addition, although we attempted to simulate a clinical diagnostic setting, a study setting can never replicate the context of the actual clinical workflow. Finally, despite having established a sizeable reader group, the small subgroup sizes limited reliable analysis by the linear mixed‐effects model. Nonetheless, we believe that this study clearly demonstrates the potential that this DL model can have on screening for STIC/STIL.

Future work to strengthen the DL model would ideally encompass broad collaboration between institutions and experts. Additional cases of STIC/STIL to further deepen the expert‐pathologist‐driven reference standard and further train the DL model could eventually lead to a classification model, capable of assisting pathologists to differentiate between STIC and STIL. To further analyze the effect on readers, a prospective trial in a clinical setting would be optimal. With the ongoing digitalization of pathology, such an approach will become more feasible over time.

In conclusion, this study evaluated the impact of a DL model designed for the automated detection of STIC/STIL lesions in WSI of fallopian tube specimens. Across a diverse group of readers, from 11 countries, professional backgrounds and varying years of working experience, we observed a significant improvement in accuracy of STIC/STIL detection and a significant decrease in slide review time. Most readers reacted positively to working with the model and indicated that they would be interested in using the model in clinical practice. We believe this model can meaningfully assist pathologists in diagnosing STIC/STIL and can optimize the diagnostic process for these lesions.

Author contributions statement

JMAB, MPS, J‐MB, MHDvB, MS, JAH and JAWML conceived and designed the study. All authors collected and assembled the data; interpreted and analyzed the data; wrote and had final approval of the manuscript; and are accountable for all aspects of the work.

Supporting information

Section S1. Readers' background information

Section S2. Individual readers' sensitivity and specificity

Section S3. Individual readers' average slide review time

Section S4. Individual readers' certainty score

Section S5. Subgroup analysis

AI‐STIC Study Group (for group authorship)

Joost Bart, Department of Pathology and Medical Biology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.

Jessica L Bentz, Department of Pathology, Dartmouth Hitchcock Medical Center, Lebanon, NH, USA.

Tjalling Bosse, Department of Pathology, Leiden University Medical Center, Leiden, The Netherlands.

Johan Bulten, Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.

Mohamed Mokhtar Desouki, Department of Pathology, Roswell Park Comprehensive Cancer Center, Buffalo, NY, USA; Jacobs School of Medicine and Biomedical Sciences, State University of New York at Buffalo, Buffalo, NY, USA.

Ricardo R Lastra, Department of Pathology, University of Chicago, Chicago, IL, USA.

Tricia A Numan, Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA.

J Kenneth Schoolmeester, Department of Laboratory Medicine and Pathology, Mayo Clinic, Jacksonville, FL, USA.

Lauren E Schwartz, University of Pennsylvania, Department of Pathology and Laboratory Medicine, Philadelphia, PA, USA.

Ie‐Ming Shih, Department of Gynecology and Obstetrics and Pathology, Johns Hopkins Medical Institution, Baltimore, MD, USA.

T Rinda Soong, Department of Pathology, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA.

Gulisa Turashvili, Department of Pathology and Laboratory Medicine, Emory University Hospital, Atlanta, GA, USA.

Russell Vang, Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, MD, USA.

Mila Volchek, Department of Anatomical Pathology, Royal Children's Hospital, Parkville, Victoria, Australia.

Riena P Aliredjo, Department of Pathology, Rijnstate Hospital, Arnhem, The Netherlands.

Heidi Kusters‐Vandevelde, Department of Pathology, Canisius Wilhelmina Hospital, Nijmegen, The Netherlands.

Data availability statement

Images are subject to various data transfer agreements. These images can be requested at the respective pathology institutions. Source code to train and assess the DL model, and data about the reference standard, are available from the corresponding author on reasonable request. The DL model will be made freely accessible online (grand-challenge.org).