Development and clinical validation of deep learning-based immunohistochemistry prediction models for subtyping and staging of gastrointestinal cancers

Abstract

Background

Immunohistochemistry (IHC) is a critical tool for tumor diagnosis and treatment, but it is time and tissue consuming, and highly dependent on skilled laboratory technicians. Recently, deep learning-based IHC biomarker prediction models have been widely developed, but few investigations have explored their clinical application effectiveness.

Methods

In this study, we aimed to create an automatic pipeline for the construction of deep learning models to generate AI-IHC (Artificial Intelligence) output using H&E whole slide images (WSIs) and compared the pathology reports by pathologists on AI-IHC versus conventional IHC. We obtained 134 WSIs including H&E and IHC pairs, and automatically extracted 415,463 tiles from H&E slides for model construction based on the annotation transfer from IHC slides. Five IHC biomarker prediction models (P40, Pan-CK, Desmin, P53, Ki-67) were developed to support a range of clinically relevant diagnostic applications across various gastrointestinal cancer subtypes, including esophageal, gastric, and colorectal cancers. The Ki-67 proliferation index was quantitatively assessed using digital image analysis.

Results

The AUCs of five IHC biomarker models ranged from 0.90 to 0.96 and the accuracies were between 83.04 and 90.81%. Additional 150 WSIs from 30 patients were collected to assess the effectiveness of AI-IHC through the multi-reader multi-case (MRMC) study. Each case was read by three pathologists, once on AI-IHC and once on conventional IHC with a minimum 2-week washout period. The results indicate that the consistency rates of pathologists in AI and conventional IHC cases were high in Desmin, Pan-CK and P40 (96.67-100%) while moderate in the P53 (70.00%). We also evaluated the T-stage through the staining of these IHC biomarkers and the consistency rate was 86.36%. Furthermore, the Ki-67 proliferation index, as reported by AI-IHC, showed a variability ranging from 17.35% ±16.2% compared to conventional IHC, with ICC of 0.415 (P = 0.015) between these two groups.

Conclusions

Here, we leveraged automatic tile-level annotations from H&E slides to efficiently develop deep learning–based IHC biomarker models, achieving AUCs between 0.90 and 0.96. AI generated IHC showed substantial concordance with conventional IHC across most markers, supporting its potential as an assistive tool in routine diagnostics.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12876-025-04045-0.

Background

Gastrointestinal (GI) cancers account for 18.8% of the global cancer incidence [1]placing a significant burden on daily healthcare practices [2]. As the gold standard for cancer diagnosis, immunohistochemistry (IHC) plays a pivotal role in the comprehensive pathological assessment of GI cancers, covering aspects from diagnosis to therapy and prognosis [3]. However, its widespread adoption is hindered by limitations such as high labor intensity, time consumption, shortage of trained professional staining technicians, and tissue costs [4].

Deep learning offers the ability to capture subtle morphological features from complex whole-slide images (WSIs) [5, 6]resenting a promising solution to the labor-intensive challenges of conventional IHC. Currently, the prediction of IHC biomarkers using AI (Artificial Intelligence) has primarily focused on cancer diagnosis, with notable achievements reported in areas such as microsatellite instability (MSI), P53 expression [7–11] and so on. Previous convolutional neural network models for p53 expression prediction reported moderate performance with mean average precision 0.731–0.754, but the division of patches from the same patients might lead to overestimation [11]. Similarly, a study [12] evaluated breast cancer Ki-67LI using a standard reference card combined with AI software, but the reliance on manual methods to establish the gold standard increased time consumption.

However, these studies primarily focused on single molecular biomarker prediction of specific digestive disease. Given the complexity of the tumor microenvironment and immune response, relying on a single molecular biomarker is insufficient for robust prognosis or therapeutic guidance [13]. Additionally, many previous studies lack real-world validation, raising concerns about their performance in standard diagnostic workflows [14].

IHC is a vital diagnostic tool in GI pathology, used for subtyping cancers and assessing prognosis. As described by the fifth edition of WHO in digestive tumors [15]P53 mutations are frequently seen in well-differentiated carcinomas and adenocarcinomas. P40 has been shown to perform with higher accuracy in distinguishing poorly differentiated adenocarcinomas from squamous cell carcinomas. In addition to these, we incorporated several biomarkers that are used in diagnostic workflows [16–19]which has not yet been explored in predictive models. Desmin helps assess submucosal invasion, Pan-CK confirms epithelial origin, and Ki-67 provides insight into tumor proliferation. Accurate prediction of multiple key biomarkers commonly used in clinical practice would significantly aid routine pathology workflows. However, due to the time-consuming nature of pathologist annotations, studies predicting multiple IHC biomarkers using standardized pipelines are still rare.

The success of AI-based methods heavily depends on the quality and scale of training data [20]. Most AI algorithms require pathologists, preferably experts, to annotate the slides, with manually delineating regions of interest. However, due to time constraints and the economic burden of labeling, images annotated by experts remain scarce [21]. To overcome this bottleneck, automated pipelines are needed to enable rapid and precise label generation, facilitating efficient model training.

In this study, we aimed to develop an automated pipeline for constructing deep learning models that generate AI-driven virtual IHC staining results directly from H&E-stained WSIs. We trained the model by retrospectively collecting 134 WSIs from Renmin Hospital of Wuhan University (WURH). The output results were visualized to enable precise biomarker prediction at the cellular level. To our knowledge, this is the first study to conduct a Multi-Reader Multi-Case (MRMC) validation comparing AI-generated and conventional IHC in real-world diagnostic scenarios. Our findings demonstrated strong concordance between AI and conventional IHC across most molecular biomarkers, highlighting the potential of AI-IHC as a valuable assistive tool in diagnostic practice.

Materials and methods

Whole-slide images preparation

We retrospectively selected data from patients with confirmed gastrointestinal (GI) cancers who underwent referral IHC testing, including esophageal squamous cell carcinoma, gastric adenocarcinoma, and colorectal adenocarcinoma. Cases were selected to represent the most prevalent histological subtypes within each anatomical site. A total of 134 WSIs—including paired H&E and IHC stains—were retrospectively collected from 73 patients at Renmin Hospital of Wuhan University. For each biomarker model, training and testing datasets were derived from non-overlapping patient cohorts. Summary statistics of WSI and tiles distribution are provided in Table S1.

The dataset encompasses five clinically meaningful IHC biomarkers: P40, Pan-CK, Desmin, P53, and Ki-67, scanned using KF-PRO-020 (KFBIO, Ningbo, China) and the Pannoramic 250 Flash Scanner (3DHISTECH, Budapest, Hungary). For P53, we included cases with overexpression, defined as more than 80% of tumor cells showing strong nuclear staining, and wild-type expression, defined as less than 80% staining or patchy nuclear expression [17]. The WSIs were segmented into non-overlapping tiles measuring 512 × 512 pixels at a magnification of 20×.

Automatic annotation of target regions

IHC-to-H&E Label Mapping by HEMnet [22].

HEMnet is a deep learning model designed for molecular transformation from histopathology images [22]. We utilized the HEMnet neural network to align the corresponding IHC and H&E WSIs, transferring the molecular labels from IHC slides to H&E slides. The registration process followed previously published methods, combining rigid (affine transformation) and non-rigid (B-spline-based) registration techniques. This allowed for correction of both global shifts and local deformations between tissue sections.

However, the requirement for the H&E and corresponding IHC slides to be adjacent limits their clinical utility. Nonetheless, the lack of adjacency between slides did not necessarily imply poor alignment. To address this, two cases were selected, and ten consecutive H&E slides of two cases were prepared and scanned into WSIs using the same scanner (KFBIO). In order to determine the range of IHC slide sequences that can be matched precisely, we calculated the mutual information of different adjacent slices. Mutual information is a statistical measure that quantifies the amount of shared information between two images, reflecting their structural similarity and alignment consistency [23]. Using the first and second adjacent slides as a reference, we compare the changes in mutual information between the first and third, the first and fourth. and the first and tenth slices in succession.

Double check by pathologists

Before cropping the IHC negative and positive tiles annotated from the HEMnet transformation of the corresponding IHC slides, a pathologist must review and verify the accuracy of the automatic annotations. The H&E whole-slide images with automated annotations were uploaded to the VGG Image Annotator (VIA, Version 2.0.8, Oxford University, UK). Using the VIA platform, a pathology expert (ZSY, with over five years of experience) could examine the complete WSIs with automatic annotations on the H&E slides and mark any errors for correction. After refining the annotations, the adjusted regions were used for tile extraction to train the model.

Model development and test

Construction of five IHC biomarker models

We obtained 134 WSIs including H&E and IHC pairs from WURH for model development and validation. The number of WSIs used for each model in training sets and testing sets were presented in the Table S1. From these WSIs, a total of 415,463 tiles were extracted from the H&E-stained slides for model construction. We tested these models by using independent WSIs. The training sets and the testing sets were non-overlapping and randomly allocated.

The models, based on semi-supervised deep learning, were trained to predict positive and negative tiles. Positive tiles represent regions with positive IHC staining, determined by thresholding the DAB channel after color deconvolution, while negative tiles represent regions with negative IHC staining. These tiles were extracted from IHC annotations, which were initially generated by HEMnet and subsequently corrected using the VIA tool. Final image tiles were then cropped from the corrected annotations. Figure 1 illustrated the entire workflow.

Fig. 1.

Overview of the IHC biomarker model development workflow. This process consisted of three major phases. First, molecular labels from immunohistochemistry (IHC) whole slide images (WSIs) were transferred to the corresponding hematoxylin and eosin (H&E) WSIs using a deep learning-based label propagation method. Second, expert pathologists manually reviewed and corrected any annotation mismatches to ensure high-quality ground truth labels. Third, the corrected labels were used to train deep convolutional neural networks capable of predicting biomarker expression directly from H&E slides. This figure illustrates the complete pipeline from raw data to model training. AI: Artificial Intelligence. IHC: Immunohistochemistry. H&E: Hematoxylin and Eosin

The architecture of model development

This study developed IHC biomarker prediction models using a Mean Teacher semi-supervised learning framework, with ResNet-50 (pretrained on ImageNet) as the backbone network. Prior to training, all H&E image tiles were stain-normalized using the Vahadane method, combined with iterative luminosity standardization, to minimize inter-slide color variability and ensure consistent staining appearance across the dataset. The student model was optimized via a combined loss function: supervised loss (binary cross-entropy for labeled data) and consistency loss (mean squared error between student and teacher predictions under stochastic perturbations). The total loss was defined as Ltotal​=Ls​+λLc​ ​, where λ linearly increased during training. The model utilized the SGD optimizer with a base learning rate of 1 × 10^− 3, weight decay of 2 × 10^− 4, and a batch size of 32. No learning rate ramp-up was applied (lrrampup = 0). Data augmentation included random rotation, flipping, color jitter (brightness = 0.1, contrast = 0.1), and Gaussian blur (σ = 0.1). Weighted sampling addressed class imbalance.

The process of Ki-67 proliferation index value calculation

The quantification of Ki-67 expression by immunohistochemistry, which is known as the Ki-67 labelling index has become a routine practice in clinical pathology to assess tumor cell proliferation. The Ki-67 proliferation index (Ki-67LI) [24] was calculated as: Ki-67LI (%) = (Number of Ki-67‑positive invasive cancer cell nuclei / Total number of invasive cancer cell nuclei) × 100. In this study, the invasive cancer areas included model-P40 positive prediction areas and model-PCK positive prediction areas (Fig. 2). Using deep learning models, we aimed to predict these molecular biomarkers expression in H&E WSIs, while also quantifying the Ki-67 proliferation index.

Fig. 2.

Ki-67 proliferation index value calculation process. The Ki-67 proliferation index is defined as the proportion of invasive cancer cell nuclei that show positive immunostaining for Ki-67, relative to the total number of invasive cancer nuclei within a given histological region. In this study, we identified invasive cancer regions using a combination of predictions from the model trained on P40 and Pan-CK. This approach enables precise localization of tumor areas and robust quantification of proliferation activity in a clinically relevant context

MRMC study for clinical application

To assess the applicability of the model, we randomly selected 150 WSIs from 30 cases including paired H&E and IHC that were not used in the model development. In this MRMC clinical trial, three pathologists (ZZ has more than 10 years of experience in gastrointestinal cancer pathology diagnosis; LW and CFF have more than five years of experience in gastrointestinal cancer pathology diagnosis.) were involved into a real-world clinical scenario to write pathology reports according to AI-IHC and Conventional-IHC, respectively. The trail consisted of two rounds involving the interpretation of AI-IHC and Conventional-IHC for pathology reporting. Using a crossover design, 30 cases were randomly and equally divided into 15 cases of AI-IHC and 15 cases of Conventional-IHC. After a washout period of two weeks, the arrangement of cases was reversed. In order to better explore the accuracy of target area in the prediction of Desmin, Pan-CK and P40, we compared the T stage of each case in the AI and conventional IHC based on the classification of pathology diagnosis. T stage evaluation was performed by three pathologists, each referring to the AJCC 8th edition [25].

In the AI-IHC group, tumor regions were determined by referring to the H&E slides and AI-generated IHC slides, while in the Conventional-IHC group, tumor regions were identified using H&E slides and conventional IHC staining. Each pathologist evaluated the slides independently. As for Ki-67 proliferation index, in the Conventional-IHC group, pathologists visually estimated the Ki-67 proliferation index based on H&E and IHC slides, while in the AI-IHC group, AI-generated Ki-67 proliferation index values were provided as reference. Three pathologists were requested to finish an online questionnaire (Table S2) after completing the MRMC trail. The baseline of MRMC study was shown in the Table 1. Besides GI cancers, a total of 30 control cases, including 22 neoplastic and 8 non-neoplastic lesions (including gastritis, polyps and intestinal metaplasia), were included to simulate a real-world diagnostic setting.

Table 1.

The baseline of MRMC

Case Number	Location	Histological type	Immunohistochemical Biomarker					Tumor depth	Pathological Stage
			PCK	P40	Desmin	Ki67 [%]	P53
1	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	90%	overexpression	T1	IA
2	Stomach	Non-atrophic gastritis	-	NA	NA	low	wild-type	NA	NA
3	Stomach	Well-differentiated adenocarcinoma	+	NA	+	70%	wild-type	T1	IA
4	Stomach	Well-to moderately differentiated adenocarcinoma	+	NA	+	60%	wild-type	T1	IA
5	Stomach	Poorly differentiated adenocarcinoma	+	NA	+	60%	wild-type	T1	IA
6	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	60%	wild-type	T1	IA
7	Esophagus	Esophagitis	NA	-	NA	low	wild-type	NA	NA
8	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	70%	overexpression	T1	IA
9	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	70%	wild-type	T1	IA
10	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	50%	overexpression	T2	IB
11	Esophagus	Ulcer	NA	-	NA	low	wild-type	NA	NA
12	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	70%	wild-type	T1	IA
13	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	40%	overexpression	T3	IIIB
14	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	70%	overexpression	T3	IIIB
15	Stomach	Well-differentiated adenocarcinoma	+	NA	+	70%	overexpression	T1	IA
16	Esophagus	Esophagitis	NA	-	NA	low	wild-type	NA	NA
17	Stomach	Polyp	-	NA	NA	low	wild-type	NA	NA
18	Stomach	Moderately to poorly differentiated adenocarcinoma	+	NA	+	90%	overexpression	T1	IA
19	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	60%	overexpression	T2	IB
20	Stomach	Moderately to poorly differentiated adenocarcinoma	+	NA	+	60%	overexpression	T3	IIIC
21	Stomach	Intestinal Metaplasia	-	NA	NA	low	wild-type	NA	NA
22	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	40%	overexpression	T1	IA
23	Stomach	Poorly differentiated adenocarcinoma	+	NA	+	60%	wild-type	T1	IA
24	Esophagus	Moderately differentiated Squamous Cell Carcinoma	NA	+	+	30%	overexpression	T3	IIIB
25	Stomach	Poorly differentiated adenocarcinoma	+	NA	+	80%	overexpression	T1	IA
26	Stomach	Atrophic gastritis	-	NA	NA	low	wild-type	NA	NA
27	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	60%	overexpression	T1	IA
28	Stomach	Moderately differentiated adenocarcinoma	+	NA	+	90%	overexpression	T2	IIIB
29	Esophagus	Moderately to poorly differentiated Squamous Cell Carcinoma	NA	+	+	30%	overexpression	T3	IIA
30	Rectal	Ulcer	-	NA	NA	low	wild-type	NA	NA

NA: Not Applicable

Ethics

This study was reviewed and approved by the Clinical Research Ethics Committee of Renmin Hospital of Wuhan University (approval number: WDRY2019-K073(X08)). The requirement for informed consent was waived by the ethics committee due to the retrospective nature of the study. All data were anonymized prior to analysis. All experiments involving human participants or human tissue samples were conducted in accordance with the principles of the Declaration of Helsinki.

Statistical analysis

The performance of the models was evaluated by accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the area under the receiver operating characteristic curve (AUC). All metrics were calculated at the tile level, using pathologist-provided annotations as the reference standard on an independent test set. In the MRMC study, for each case, we assessed whether each pathologist’s categorized diagnoses of five molecular biomarkers, as reported by both AI-IHC and Conventional-IHC, were consistent. The calculation was based on the agreement rate in the test cases. The agreement between AI-IHC and Conventional-IHC of three endoscopists was calculated using the Kappa by SPSS (Version 27, IBM, USA). Performance metrics of Ki-67 proliferation index by three pathologists between AI-IHC and Conventional-IHC were compared using intraclass correlation coefficient (ICC) and Bland-Altman analysis. Data visualization was conducted with GraphPad Prism, and model development utilized Python in Jupyter Notebook environment. Annotation verification employed VGG Image Annotator (VIA 2.0.8), and final figures were prepared using Microsoft PowerPoint 365.

Result

The efficiency of automatic annotations

Using the mutual information values of the 1st to 2nd slides (0.658 for WSIs-1; 0.716 for WSIs-2) as a reference, the mutual information values of the 1st to 3rd, 1st to 4th. 1st to 10th in succession show a negligible decrease, with the decline ranging from 0.054 to 0.132 for WSIs-1 and 0.067 to 0.135 for WSIs-2 (Fig. S1). Consequently, our experiment focused on training and testing with pairs of HE and IHC slides within a contiguous series of up to 10 slides. After transforming the matched slides into molecular biomarker labels through deep learning, pathologists reviewed and corrected the incorrectly matched tiles on the online annotation software (VIA platform). During this process, the time of pathology reviewing ranged from 14 ± 8 min per WSI. Compared with previously reported manual annotation times ranging from 92.3 to 8033 min per slide [26]our approach substantially reduced the annotation time while still producing precisely labeled tiles that captured every cellular structural detail on the WSI.

Enhancing accuracy through pathologist double-checks

To compare how pathology review can increase the performance of models, we also trained five IHC biomarker models without any pathologists’ corrections. The accuracy improvement of five models ranged from 2.50 to 13.88%, Table S3 presented the detailed results.

Validation of five IHC biomarkers models

Trained based on the precise labeled tiles, the accuracy of five IHC biomarkers models in the testing sets for predicting PAN-CK, P40, Desmin, P53, and Ki-67 reached 90.81% (95% CI: 90.38-91.23%), 88.66% (95% CI: 88.25-89.06%), 89.16% (95% CI: 88.65-89.95%), 83.04% (95% CI: 82.51-83.55%), and 86.77% (95% CI: 86.35-87.17%), respectively (Table 2). The AUC of five biomarkers models for PAN-CK, P40, Desmin, P53, and Ki-67 was 0.961, 0.954, 0.956, 0.898, 0.948, respectively. The receiver operating characteristic curves of five IHC biomarkers were shown in the Fig. 3. The IHC biomarker models predicted positive tiles in red, color-coded according to confidence interval values, and negative tiles in blue on the H&E whole-slide images. The predictive outputs, compared with the golden standard IHC, were illustrated in Fig. 4. The confusion matrix of five IHC biomarkers was shown in the Fig. S2.

Table 2.

The performance of five immunohistochemical biomarker models in testing sets

Model-Biomarker	Accuracy,95%CI (%)	Sensitivity,95%CI (%)	Specificity,95%CI (%)	PPV,95%CI (%)	NPV,95%CI (%)
Model-PAN-CK	90.81(90.38–91.23)	92.40(91.82–92.95)	89.35(92.19–93.27)	88.86(88.19–89.50)	92.75(92.19–93.27)
Model-P40	88.66(88.25–89.06)	91.64(90.96–92.27)	87.39(86.87–87.89)	75.56(74.63–76.47)	96.09(95.76–96.39)
Model-Desmin	89.16(88.65–89.65)	91.44(90.61–92.20)	88.08(87.44–88.69)	78.30(77.20-79.36)	95.63(95.20-96.03)
Model-P53	83.04(82.51–83.55)	75.44(74.36–76.49)	86.52(85.94–87.08)	71.88(70.78–72.95)	88.52(87.97–89.05)
Model-Ki67	86.77(86.35–87.17)	89.63(88.78–90.43)	86.04(85.56–86.50)	61.96(60.87–63.04)	97.03(96.77–97.27)

PPV: Positive Predictive Value. NPV: Negative Predictive Value. CI: Confidence Interval

Fig. 3.

Receiver Operating Characteristic (ROC) curves for the five biomarker models. The ROC curves represent the classification performance of deep learning models trained to predict expression of Pan-CK, P40, Desmin, P53, and Ki-67 on H&E slides. Each curve reflects the model’s ability to distinguish between positive and negative regions, with the Area Under the Curve (AUC) serving as the main performance metric

Fig. 4.

Visualization of AI-based virtual IHC predictions and comparison with conventional IHC slides. The first row presents representative H&E WSIs corresponding to each biomarker. The second row shows the deep learning model’s predictions: red indicates tiles predicted as positive (with varying intensity based on confidence levels), and blue indicates negative predictions. The third row shows the corresponding conventional IHC-stained WSIs, serving as the ground truth. This side-by-side comparison demonstrates the visual alignment and interpretability of the AI-generated IHC with conventional IHC. ACC: Accuracy. AI: Artificial Intelligence. IHC: Immunohistochemistry. CI: Confidence Interval. H&E: Hematoxylin and Eosin

The clinical validation in MRMC study

Three pathologists wrote reports of five IHC biomarkers in 30 cases according to Conventional-IHC WSIs and AI-IHC WSIs. Compared with the Conventional-IHC result, the AI-IHC reports’ consistency rate for each IHC biomarker reached: Desmin (100.00%, 66/66), Pan-CK (98.33%, 59/60), P53 (70.00%, 63/90), P40 (96.67%, 29/30). The consistency rate of T stage was 86.36% (57/66). For T1 stage in the report’s description, the highest consistency between conventional and AI-IHC was observed (97.56%, 40/41), with only one case reported incorrectly. Details of T stage classification was shown in the confusion matrix of Fig. S3.

Pathology agreements between Conventional-IHC and AI-IHC were high in Pan-CK, P40, Desmin. The Kappa of each IHC biomarker model was: model-PCK (Kappa = 0.955), model-P40 (Kappa = 0.918), model-P53 (Kappa = 0.379), model-Desmin of T stage classification (Kappa = 0.735). According to commonly accepted criteria, Kappa values between 0.61 and 0.80 indicate substantial agreement, and values above 0.80 represent almost perfect agreement, while values below 0.40 reflect poor agreement. Based on these benchmarks, Pan-CK, P40, and Desmin models achieved substantial to almost perfect agreement, whereas the P53 model showed relatively lower consistency. Each pathologist’s result was visualized in the Fig. 5 in detail.

Fig. 5.

MRMC (Multi-Reader Multi-Case) results from three pathologists. The figure summarizes the individual T-stage assessments made by three experienced pathologists (P1: ZZ, P2: CFF, P3: LW) for Pan-CK, P40, P53, and Desmin. Assessments were made using both AI-generated IHC and conventional IHC slides. Red represents biomarker positivity, while blue indicates negativity. The bottom row shows the original T-stage reported in clinical pathology reports, enabling a comparative analysis of diagnostic consistency. Ori-Report: Original- Report

The quantification of Ki-67 in prediction models

As for the prediction and quantification of Ki-67, the proliferation index reports according to AI-IHC presented a difference ranging from 17.35%±16.2%. The consistency between the two assessments (Conventional-IHC and AI-IHC) showed an ICC of 0.415 (P = 0.015). Differences between the two assessments (Conventional-IHC and AI-IHC) were visualized in detail by Bland Altman in the Fig. S4.

Comparing the satisfaction level of conventional IHC and AI-IHC

The result showed that in pathologists’ view, the accuracy (8.67 vs. 5.67, P = 0.184) and trust (8.67 vs. 5.67, P = 1.000) of Conventional-IHC with AI-IHC had no significant difference. All three pathologists agreed AI-IHC would be economical, tissue saving and assisted for diagnosing in clinical work.

Discussion

In this study, we developed an automatic pipeline for model training of five clinically IHC biomarkers in GI cancers. The AI-IHC outputs exhibited promising agreement with conventional IHC techniques across most biomarkers in the MRMC validation. The integration of real-world clinical data and pathologist assessments strengthened the evidence for practical utility of these models [27]. To our knowledge, this is the first study to conduct an MRMC trail validating the real-world applicability of IHC prediction models within routine histopathological workflows.

Although conventional IHC remains fundamental to tissue diagnosis, it is time-consuming and tissue-consuming, especially when multiple markers are needed in small biopsies [28, 29]. This can limit comprehensive diagnostic evaluation. While multiplex IHC techniques have been developed to reduce tissue use, their complexity and cost restrict large-scale clinical adoption [30]. Therefore, alternative approaches such as AI-IHC are needed to reduce reliance on conventional staining.

Recent progress in digital pathology and deep learning has facilitated visual analysis of histology slides [31–33]. Most studies focused on diagnosis or identification of molecular biomarkers that could be associated with immune checkpoint inhibitors, such as MSI [7] PD-L1 status [34] and inflammatory genes [35]. However, routine biomarkers essential to everyday diagnostic practice remain underexplored. Our study targeted five routinely used biomarkers (Pan-CK, P40, Desmin, P53, and Ki-67) and achieved high AUC values ranging from 0.90 to 0.96, emphasizing the potential of AI-IHC in routine settings.

Furthermore, we visualized predicted IHC-positive regions using graded red color intensities corresponding to prediction confidence levels, which assisted pathologists in understanding the model’s decision-making process and enhanced clinical usability. Although our model was developed for GI cancers and currently focuses on binary classification, the framework is generalizable. A potential extension is to address IHC markers that require quantitative or semi-quantitative interpretation, such as Her2 in breast cancer [36]by training an IHC-based multi-class classifier and transferring the resulting class-specific labels to H&E images for supervised learning.

A key strength of our work is the MRMC clinical validation. While numerous models have been developed for routine pathology tasks [37]the challenge persists in pathologists’ acceptance and utilization of these tools [38, 39]. In our trail, three pathologists participated in reporting pathology findings using both AI-IHC and conventional IHC slides, while they were not involved in the model development. During the study, it can be proved that AI-IHC demonstrated high concordance with conventional IHC for Pan-CK, P40, and Desmin. Although pathologists were instructed to assess slides based solely on diagnostic criteria, their awareness of image origin might have introduced subtle interpretative biases. Moreover, the use of T-stage as a clinical evaluation endpoint may also be limited, as it can be determined using H&E slides and is seldom guided directly by IHC results in standard practice.

To investigate application scenarios, we conducted a questionnaire to assess three pathologists’ attitudes towards conventional IHC versus AI-IHC. On the one hand, all participants acknowledged the economic benefits of AI-IHC but maintained a cautious stance toward the reliability of AI-generated IHC outputs. While the system was well received for its practicality and potential clinical value, further validation is required to determine whether it can reliably replace conventional IHC in routine diagnostic practice. Integrating AI-IHC into workflows requires addressing challenges in trust, performance, and standardization [40]. Broader adoption may benefit from expert consensus, large-scale trials, and real-time feedback. On the other hand, pathologists all noted the false positives in both conventional IHC and AI-IHC, particularly for P40 and Pan-CK. To address this, our AI-IHC models were trained using high-quality, pathologist-verified IHC annotations to minimize such artifacts. during label transfer using HEMnet, we applied thresholding to the DAB signal on IHC images to minimize the inclusion of non-tumor cells. To further mitigate this limitation, a potential future direction could involve a two-stage framework: first, applying a pre-filtering deep learning model to exclude ambiguous regions, followed by high-specificity label transfer for biomarker prediction.

Nevertheless, several limitations should be acknowledged. The P53 model showed relatively lower performance (Kappa = 0.379), potentially due to the weak correlation between protein expression and morphological features, as well as reduced interpretative consistency. While pathologists demonstrated strong agreement when assessing conventional IHC (Kappa = 0.955), this agreement decreased for AI-generated IHC (Kappa = 0.656), suggesting that virtual staining may introduce interpretive ambiguity. Improving the interpretability and trustworthiness of AI outputs could be a key step toward broader clinical adoption. Future work should focus on training with larger, more diverse datasets to enhance generalizability. In addition, transitioning from classification to segmentation-based models may allow better capture of multidimensional IHC expression features on H&E slides, not limited to morphological cues alone. Furthermore, the current P53 model only distinguishes between wild-type and overexpression patterns, without identifying the null-type phenotype, thereby limiting its clinical applicability in certain cases.

The retrospective, single-center design limits the generalizability of the findings. Potential solutions include multi-center collaboration, use of public datasets, and transfer learning to adapt the models to varied staining protocols and patient populations. As for Ki-67, the intraclass correlation coefficient (ICC) between AI and manual scores was 0.415, indicating moderate agreement. This variability aligns with previous studies reporting low interobserver reproducibility for Ki-67 [41, 42], highlighting the inherent challenge of evaluating this marker. Additionally, the level of trust pathologists placed in the AI-IHC results may also have influenced variability. In the future, improving the interpretability of the model and using AI to support the development of more consistent scoring criteria may help reduce this variation. Nevertheless, the quantitative capacity of AI may offer a path to developing new Ki-67 stratification strategies that better correlate with clinical outcomes. Looking ahead, the application of artificial intelligence in digital pathology is expected to evolve toward real-time diagnostic assistance, enabling interactive interpretation through both visual and language-based interfaces [43, 44]. With the advancement of computing power, future integration of high-performance AI systems may further enhance diagnostic efficiency and streamline pathology workflows.

Conclusions

Generally, we construct an automatic pipeline for five widely used biomarkers models training. The performance of AI-IHC is great in the test sets and we visualize the AI output at the tile level. In the following MRMC clinical trial, AI-IHC models exhibited promising agreement with conventional IHC techniques across most IHC biomarkers.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

AI

Artificial intelligence

GI

Gastrointestinal

IHC

Immunohistochemistry

ICC

Intraclass Correlation Coefficient

MRMC

Multi-Reader Multi-Case

MSI

Microsatellite Instability

NPV

Negative Predictive Value

PPV

Positive Predictive Value

WSIs

Whole Slide Images

WURH

Renmin Hospital of Wuhan University

Author contributions

HGY had the idea and supervised the overall study. JXW designed and did the experiments. HGY, LLW supervised the experiments. JXW and SYZ developed the system. SYZ, MD, ZHD, ZZ, DDC, WL were involved in data collection. JXW wrote the report. JXW and JL analyzed the data. HGY was responsible for the overall content as guarantor. All authors approved the final version of the report.

Funding

This work was partly supported by the Key Research and Development Program of Hubei Province (grant no. 2023BCB153, to Honggang Yu) and National Natural Science Foundation of China-Youth Science Fund Project (grant number 82202257), National Key R&D Program of China (Grant Nos. 2022YFC2505106), The Interdisciplinary Innovative Talents Foundation from Renmin Hospital of Wuhan University (JCRCFZ-2022-010) (to Lianlian Wu).

Data availability

Individual de-identified participant data that underline the results reported in this article can be shared with investigators for research purposes. Access to the data can be requested from the first corresponding author, yuhonggang@whu.edu.cn. The algorithm is publicly available on GitHub at: https://github.com/endo-angel/Virtual-immunohistochemistry-based-on-AI.

Declarations

Ethics approval and consent to participate

This study was reviewed and approved by the Clinical Research Ethics Committee of Renmin Hospital of Wuhan University (approval number: WDRY2019-K073(X08)). The requirement for informed consent was waived by the ethics committee due to the retrospective nature of the study. All data were anonymized prior to analysis. All experiments involving human participants or human tissue samples were conducted in accordance with the principles of the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.