TiLCA-Net: nuclei segmentation and classification in pathological images via task-specialized attention and efficient decoding

Abstract

Background

Nuclei segmentation and classification in whole-slide images (WSIs) are critical for tumor diagnosis and treatment. Existing methods often face multi-task feature conflict and computational redundancy. This study aimed to develop an efficient multi-task framework that balances segmentation accuracy and inference efficiency.

Methods

To seek an improved balance between segmentation accuracy and efficiency, this study proposes TiLCA-Net, a network that integrates multi-scale attention mechanisms with a lightweight decoding strategy to optimize comprehensive model performance. First, a Task-Specialized Attention Integrated Encoder (TSEncoder) is designed to decouple features by assigning spatial, global, and channel attention at multiple levels, thereby mitigating negative transfer. Second, we introduce an iterative cross-enhancement (ICE) module, which utilizes closed-loop iteration and median pooling to enhance global geometric consistency and robustness. Finally, a Lightweight Context Aggregation Decoder (LCA-Decoder) is introduced, replacing redundant structures with multi-scale aggregation to substantially reduce computational cost while preserving high-frequency details.

Results

Experiments on CoNSeP, MoNuSeg, and CervCell-Seg demonstrate that TiLCA-Net achieves a favorable balance between segmentation performance and computational efficiency. TiLCA-Net achieved mean panoptic quality (PQ) scores of 0.528, 0.578, and 0.592 on CoNSeP, MoNuSeg, and CervCell-Seg, respectively, compared with 0.504, 0.563, and 0.578 for HoVer-Net. These results indicate PQ improvements of 0.024, 0.015, and 0.014 on the three datasets, respectively. Patch-level benchmarking showed that TiLCA-Net required fewer floating point operations per second (FLOPs) and achieved higher batch throughput than HoVer-Net, but with higher single-patch latency and peak graphics processing unit (GPU) memory.

Conclusions

Through feature decoupling and efficient decoder reconstruction, TiLCA-Net provides a practical accuracy-efficiency trade-off for patch-level nuclei analysis in computational pathology.

Introduction

In modern clinical practice driven by precision medicine, histopathological examination based on tissue biopsy remains central to tumor diagnosis, grading, and prognostic evaluation (1,2). Although radiomics provides valuable macro-level tumor assessment, histopathology offers direct microscopic evidence of tissue morphology. With the advancement of computational pathology, whole-slide images (WSIs) have become an important medium for capturing the multidimensional information of the tumor microenvironment (TME) at gigapixel resolution (3). WSIs provide a data bridge for analyzing deep pathological features, ranging from tumor atypia to the spatial heterogeneity of stromal components (4).

Within this microscopic ecosystem, nuclei provide essential information for clinical decision-making. Nuclear morphology, mitotic activity, and spatial distribution are closely associated with pathological progression (5). For example, tumor epithelial atypia supports malignancy grading (6), normal epithelial cells reflect baseline tissue organization (7), spindle cells are important for soft-tissue tumor differentiation (8), and inflammatory cell infiltration can indicate the immune status of the TME (9). Therefore, accurate and high-throughput nuclei instance segmentation and classification are essential for building intelligent computer-aided diagnosis systems.

In recent years, nuclei segmentation and classification have been extensively investigated. Early methods mainly relied on thresholding and watershed-based image processing strategies (10,11). With the development of deep learning, convolutional neural network (CNN)-based models became the dominant solution. U-Net and nnU-Net enabled pixel-level nuclei segmentation (12,13), while HoVer-Net introduced horizontal and vertical distance maps to improve the separation of overlapping nuclei and became a widely used benchmark for simultaneous nuclei segmentation and classification (14). More recently, Transformer-based methods, such as CellViT and UN-SAM, have enhanced global modeling ability, although their computational complexity can limit high-throughput WSI analysis (15-17).

Traditional nuclei segmentation methods were further extended through marker-controlled watershed, active contours, and hand-crafted morphological features (18-22). However, these methods are sensitive to staining variation, stromal noise, and nuclear adhesion. Deep learning-based instance segmentation methods have improved robustness through detection-based, contour-aware, and geometry-guided designs. For example, Mask R-CNN and its histopathology-specific variants use a detect-then-segment paradigm (23,24), whereas StarDist, DCAN, and BESNet improve instance separation by modeling nuclear shape or boundary information (25-27). In addition to segmentation, nuclei classification is important for characterizing the TME and supporting pathological assessment (28,29). Recent studies have incorporated classification into end-to-end nuclei analysis frameworks to reduce error propagation from cascaded pipelines (30-34). Multi-task learning has therefore become a common strategy in computational pathology (35-37). However, hard parameter sharing may cause feature conflicts between tasks with different representation preferences. SMILE improves task isolation by using independent encoders and cost-sensitive learning, although the use of multiple encoders may increase computational cost (38).

Attention mechanisms have also been introduced to enhance discriminative feature representation in medical image analysis (39-43). These methods help suppress redundant background information and emphasize task-relevant structures. Nevertheless, a single uniform attention design may be insufficient for nuclei panoptic analysis because different subtasks depend on different types of information: segmentation requires fine boundary and texture cues, instance separation benefits from global geometric consistency, and classification relies more on high-level semantic representations. Despite these advances, practical nuclei analysis in WSIs still faces three major challenges. First, multi-task feature conflict may lead to negative transfer when segmentation, instance separation, and classification share the same representation. Second, complex tissue backgrounds, staining variation, red blood cells, and necrotic regions can weaken geometric consistency and interfere with instance separation. Third, dense decoder branches and redundant feature fusion increase computational cost, limiting deployment in resource-constrained computational pathology settings.

To address these issues, this study proposes TiLCA-Net, a computationally efficient multi-task network that improves nuclei segmentation and classification through feature decoupling and lightweight structural reconstruction. The main contributions of this paper are summarized as follows:

1. Proposal of a Task-Specialized Attention Integrated Encoder (TSEncoder): Addressing the issue of multi-task feature conflict, we adopt a multi-scale attention injection strategy. By deploying spatial, dynamic global, and channel attention at different encoder levels, this module improves the decoupling of features required for segmentation, regression, and classification tasks.

2. Design of an iterative cross-enhancement (ICE) module: to enhance model robustness in noisy environments, we introduce a closed-loop iteration mechanism combined with a median pooling strategy. This enhances the global geometric consistency of features at a low computational cost, improving the model’s adaptability to noise.

3. Construction of a Lightweight Context Aggregation Decoder (LCA-Decoder): To mitigate computational load, we introduce an efficient decoding structure to replace heavy traditional decoders. Using hierarchical feature aggregation, this design reduces theoretical computational cost while preserving high-frequency details for precise segmentation. We present this article in accordance with the CLEAR reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-0938/rc).

Methods

Overview of the model architecture

To address the challenges of hematoxylin and eosin (H&E)-stained pathological images, including dense nuclear stacking, morphological heterogeneity, and complex background noise, we propose TiLCA-Net for integrated nuclei instance segmentation and classification. As illustrated in Figure 1, TiLCA-Net follows a U-Net-like encoder-decoder paradigm and consists of a shared stem, three task-specialized encoder branches, the ICE Module, the LCA-Decoder, and a branch merging/post-processing module.

Figure 1.

Overview of the TiLCA-Net architecture. CAB, channel attention block; CCA, Criss-Cross Attention; EULAM, Efficient Upsampling Large-Kernel Attention Module; GGCA, global guided context attention; ICE, iterative cross-enhancement; MedSCA, Median-based Spatial-Channel Attention; MS-CFE, Multi-Scale Context Feature Enhancement; NC, nuclear classification; NI, nuclear instance; NS, nuclear segmentation; SAB, spatial attention block.

The network contains three functionally complementary task branches. The nuclear segmentation (NS) branch generates binary semantic maps to distinguish nuclei from the background. The nuclear instantiation (NI) branch predicts horizontal and vertical distance maps for separating adjacent nuclei. To strengthen long-range geometric modeling in this branch, dynamic global guided context attention (GGCA) is embedded into its encoder pathway. The nuclear classification (NC) branch produces multi-class probability maps using high-level semantic features to distinguish different nuclear types.

After obtaining the outputs of the NS, NI, and NC branches, we used a two-stage marker-controlled watershed strategy to generate final nuclear instances (14,38). Initial markers were extracted from the gradient magnitude of the horizontal and vertical distance maps predicted by the NI branch and constrained by the foreground mask from the NS branch. A second watershed refinement based on distance transformation was then applied to reduce over-segmentation in large or fragmented nuclei. Finally, each nuclear instance was assigned a category label by majority voting over the NC branch predictions within the corresponding instance mask.

TSEncoder

ResNet-34 is adopted as the encoder backbone, and the proposed TSEncoder contains three parallel task-specialized branches (Figure 1). To align feature extraction with task-specific requirements, different attention modules are inserted at different encoder levels. In the NS branch, spatial attention blocks (SAB) are placed after shallow features (d0) to enhance boundary and texture cues. In the NI branch, GGCA is embedded at the mid-level feature (d2) to model long-range spatial dependencies and improve geometric consistency. In the NC branch, channel attention blocks (CAB) are applied before the high-level semantic feature (d3) to recalibrate channel responses for nuclear type classification.

The TSEncoder adopts a hybrid parameter-sharing strategy. The initial conv0 and max-pooling layers are shared to maintain consistent low-level inputs, whereas the residual blocks (d0–d3) are trained independently in each branch to reduce inter-task interference. Each attention module is applied only to its corresponding branch, as formulated in Eqs. [1-3].

$$\begin{array}{c}{F}_{SAB}=F\odot \sigma (Con{v}_{7\ast 7}([Av{g}_c(F);Ma{x}_c(F)]))\end{array}$$ [1]

$$\begin{array}{c}{F}_{GGCA}=Reshape(F_g\odot \sigma (T(Av{g}_h(F_g))+T(Ma{x}_h(F_g)))\odot \sigma (T(Av{g}_w(F_g))+T(Ma{x}_w(F_g))))\end{array}$$ [2]

$$\begin{array}{c}{F}_{CAB}=F\odot \sigma (MLP(Av{g}_g(F))+MLP(Ma{x}_g(F)))\end{array}$$ [3]

Where F denotes the input feature map of the corresponding branch, $F_g$ denotes the grouped feature representation in GGCA, and σ denotes the sigmoid activation function. T(·) denotes the shared 1×1 convolutional transformation used in GGCA. The symbol ⊙ denotes element-wise multiplication, [;] denotes channel-wise concatenation, and + denotes element-wise addition. $Av{g}_c$ and $Ma{x}_c$ indicate channel-wise average and maximum pooling used in SAB, while $Av{g}_g$ and $Ma{x}_g$ indicate global average and maximum pooling used in CAB. In GGCA, $Av{g}_h$/$Ma{x}_h$ and $Av{g}_w$/$Ma{x}_w$ denote directional pooling along the height and width dimensions, respectively. $F_{SAB}$, $F_{GGCA}$, and $F_{CAB}$ are the attention-reweighted feature outputs of the NS, NI, and NC branches, respectively. Finally, the deep features (d3) from all three branches are individually fed into the ICE module for global context enhancement before being forwarded to the LCA-Decoder.

ICE module

The ICE module is positioned between the TSEncoder and the LCA-Decoder to enhance global context modeling and robustness to local noise. Figure 2A shows the overall ICE structure, which follows a three-stage refinement process: global localization by the first Criss-Cross Attention ($CC{A}_1$) block, robust enhancement by the Median-based Spatial-Channel Attention (MedSCA) block, and global re-localization by the second Criss-Cross Attention ($CC{A}_2$). The mathematical formulation is given as:

Figure 2.

Architecture of the ICE module. (A) Overview of the ICE module; (B) CCA block; (C) MedSCA module. CCA, Criss-Cross Attention; Conv, convolution; DW, depth-wise; ICE, iterative cross-enhancement; MedSCA, Median-based Spatial-Channel Attention; MLP, multilayer perceptron.

$$\begin{array}{c}{z}_1=x+{\gamma }_1\cdot (A_H^1{V}_H^1+A_W^1{V}_W^1)\end{array}$$ [4]

$$\begin{array}{c}{z}_2=Con{v}_{1\times 1}(Con{v}_{1\times 1}\left(\sum _{i=1}^n{D}_i\left(DWCon{v}_{5\times 5}(A_c\odot u)\right)\right)\odot (A_c\odot u))\end{array}$$ [5]

$$\begin{array}{c}y=z_2+{\gamma }_2\cdot (A_H^2{V}_H^2+A_W^2{V}_W^2)\end{array}$$ [6]

Where x denotes the deep feature output from each TSEncoder branch. In Eq. [4], $A_H^1$ and $A_W^1$ denote the height- and width-direction Criss-Cross Attention (CCA) maps generated by $CC{A}_1$, $V_H^1$ and $V_W^1$ are the corresponding value projections, and ${\gamma }_1$ is a learnable scaling parameter. In Eq. [5], $u=GELU\left(Con{v}_{1\times 1}\left(z_1\right)\right)$ denotes the projected feature, and $A_c=\sigma \left(MLP\left(AvgP\left(u\right)\right)\right)+\sigma \left(MLP\left(MaxP\left(u\right)\right)\right)+\sigma \left(MLP\left(MedianP\left(u\right)\right)\right)$ denotes the tri-pooling channel attention map. $DWCon{v}_{5\times 5}$ denotes the initial 5×5 depthwise convolution, $D_i$ denotes the i-th multi-scale depthwise convolution branch, and ⊙ denotes element-wise multiplication. In Eq. [6], $A_H^2$ and $A_W^2$, $V_H^2$ and $V_W^2$, and ${\gamma }_2$ have the same meanings as in $CC{A}_1$ but are computed in $CC{A}_2$. Thus, ICE performs closed-loop iteration through global localization, robust enhancement, and relocalization via global consistency re-aggregation.

CCA (see Figure 2B): following Huang et al. (43), CCA approximates full-image self-attention through criss-cross operations, reducing the computational complexity from O((HW)²) to O(HW(H+W)) while capturing long-range dependencies.

Median-based Robust Dual Attention Block (MedSCA, see Figure 2C): MedSCA combines tri-pooling channel attention with multi-scale spatial attention. In the channel branch, global average pooling, max pooling, and median pooling are processed by a shared multilayer perceptron (MLP) and summed to generate the channel attention map. Median pooling is introduced to reduce the influence of outlier responses caused by staining artifacts or local noise. The tri-pooling channel attention is formulated in Eqs. [7,8]:

$$\begin{array}{c}{f}_c=\sigma (MLP(AvgP(f)))+\sigma (MLP(MaxP(f)))+\sigma (MLP(MedianP(f)))\end{array}$$ [7]

$$\begin{array}{c}{f}_1=f_c\odot f\end{array}$$ [8]

Where AvgP, MaxP, and MedianP denote global average pooling, global max pooling, and global median pooling, respectively. The shared MLP is applied independently to each descriptor. The symbol + denotes element-wise summation, and ⊙ denotes element-wise multiplication. The resulting descriptors are summed to generate the channel attention map.

In the spatial branch, a 5×5 depthwise separable convolution first extracts local spatial information. The resulting feature is then processed by multiple depthwise convolution branches with different kernel shapes to aggregate cross-scale context. The branch outputs are summed, projected by a 1×1 convolution, and applied to the channel-refined feature map as spatial attention, as formulated in Eqs. [9,10]:

$$\begin{array}{c}{f}_s=\sum _{i=1}^n{D}_i\left(DWCon{v}_{5\times 5}\left(f_1\right)\right)\end{array}$$ [9]

$$\begin{array}{c}{f}_2=Con{v}_{1\times 1}\left(f_s\right)\odot f_1\end{array}$$ [10]

Where $DWCon{v}_{5\times 5}$ denotes the initial 5×5 depthwise separable convolution, and $D_i$ denotes the i-th multi-scale depthwise separable convolution branch. The summation in Eq. [9] indicates element-wise addition of the multi-scale branch outputs.

LCA-Decoder

To reduce decoder redundancy in multi-task U-Net-like models, we introduce the LCA-Decoder. Figure 3A shows the overall cascaded decoder structure. The Efficient Upsampling Large-Kernel Attention Module (EULAM) block and its embedded large-kernel grouped attention gate (LGAG) are shown in Figure 3B,3C, respectively, whereas the MS-CFE block is shown in Figure 3D. The decoder workflow is formulated as:

Figure 3.

Architecture of the LCA-Decoder. (A) Overview of the LCA-Decoder; (B) EULAM block; (C) LGAG block; (D) MS-CFE block. Feature-map dimensions are annotated to show the progressive resolution changes across decoder stages. BN, batch normalization; CAB, channel attention block; Conv, convolution; DW, depth-wise; EULAM, Efficient Upsampling Large-Kernel Attention Module; LCA, lightweight context aggregation; LGAG, large-kernel grouped attention gate; MS-CFE, Multi-Scale Context Feature Enhancement; SAB, spatial attention block.

$$\begin{array}{c}{F}_{out}^i=F_{Conv1\ast 1}(F_{MS-CFE}(F_{EULAM}(F_{in}^i,F_{skip}^i)))\end{array}$$ [11]

Where $F_{in}^i$ and $F_{skip}^i$ denote the input features of the i-th layer and the encoder skip-connection features, respectively.

Multi-Scale Context Feature Enhancement (MS-CFE)

MS-CFE refines features through channel and spatial attention, adjusts the channel dimension with a 1×1 convolution, and then applies three parallel depthwise convolution branches with 3×3, 5×5, and 7×7 kernels. The multi-scale outputs are fused through element-wise multiplication, channel shuffle, and pointwise projection, followed by a residual connection to preserve input information.

EULAM

EULAM restores spatial resolution using bilinear interpolation followed by a 3×3 depthwise convolution. It then uses LGAG to recalibrate skip features from the encoder. LGAG employs large-kernel group convolutions to enlarge the effective receptive field with limited parameter overhead and generates an attention map to fuse decoder and skip features. The detailed formulations are provided in Eqs. [12-14]:

$$\begin{array}{c}{F}_{feat}=Con{v}_{1\ast 1}(BN(ReLU(DWCon{v}_{3\ast 3}(U{p}_{\ast 2}(F_{in}^i)))))\end{array}$$ [12]

$$\begin{array}{c}{F}_{out}^i=(F_{feat}⊚M_{att})+F_{up}\end{array}$$ [13]

$$\begin{array}{c}{M}_{att}=\sigma (Con{v}_{1\ast 1}(ReLU(BN(GroupCon{v}_{3\ast 3}(F_{feat}))+BN(GroupCon{v}_{3\ast 3}(F_{skip}^i)))))\end{array}$$ [14]

Where $F_{in}^i$ and $F_{out}^i$ denote the input and output, respectively, while M_att represents the attention map generated by the LGAG for dynamic calibration. This process fuses the current decoding features ($F_{feat}$) with the encoder-derived skip features ($F_{skip}^i$).

Loss function

To jointly optimize nuclei segmentation, instantiation, and classification, we define a multi-task loss composed of three branch-specific terms. For the NS branch, binary cross entropy loss and dice loss are used to balance pixel-wise classification and regional overlap. For the NI branch, mean squared error loss and gradient consistency loss are used to supervise the horizontal and vertical distance maps and preserve boundary geometry. For the NC branch, cost-sensitive loss and alpha-dice loss from SMILE (38) are adopted to reduce the influence of class imbalance. The total loss is formulated as:

$$\begin{array}{c}{L}_{total}={\lambda }_{seg}(L_{ce}+L_{dice})+{\lambda }_{inst}(L_{mse}+L_{msge})+{\lambda }_{cls}(L_{cs}+L_{ad})\end{array}$$ [15]

Where ${\lambda }_{seg}$, ${\lambda }_{inst}$, and ${\lambda }_{cls}$ denote the weights of the segmentation, instantiation, and classification losses, respectively. $L_{ce}$ and $L_{dice}$ are the losses for the NS branch; $L_{mse}$ and $L_{msge}$ are the losses for the NI branch; and $L_{cs}$ and $L_{ad}$ are the losses for the NC branch. The classification loss combines a cost matrix for minority-class misclassification with batch-wise class weighting to improve supervision for rare nuclear categories.

Datasets

Public benchmark: CoNSeP and MoNuSeg

We evaluated the model’s generalization performance on two public benchmarks: CoNSeP and MoNuSeg. The CoNSeP dataset (44) contains 41 H&E-stained colorectal cancer image tiles acquired at 40× magnification, each with a resolution of 1,000×1,000 pixels. While the original annotations encompass seven nuclear types, to maintain comparative consistency with state-of-the-art (SOTA) methods and align with the subsequent four-class configuration of our cervical cancer task, we followed the authors’ suggested category merging scheme: normal and malignant epithelial nuclei were merged into the Epithelial class; fibroblasts, endothelial cells, and muscle cells were consolidated into the Spindle class; and inflammatory cells alongside a small number of unclassified dead/miscellaneous cells were retained as Inflammatory and Miscellaneous, respectively. Consequently, the post-consolidation class distribution (illustrated in Figure 4A) comprises Epithelial (5,537, 36%), Spindle (5,706, 37%), Inflammatory (3,941, 25%), and Miscellaneous (371, 2%) nuclei. We also employed MoNuSeg (45), a public multi-organ H&E benchmark, to evaluate nuclei instance segmentation and cross-organ generalization. The experimental collection comprised 37 annotated training images and 14 evaluation images, each with a resolution of 1,000×1,000 pixels. Because MoNuSeg provides nuclear instance annotations without nuclear-type labels, it was used only for segmentation evaluation using Dice, Aggregated Jaccard Index (AJI), DQ, SQ, and PQ. Using a patch size of 540×540 pixels, a stride of 164 pixels, and mirror padding, we obtained 1,813 training patches and 686 evaluation patches.

Figure 4.

Proportion of different cell types in (A) CoNSeP and (B) CervCell-Seg datasets.

Private clinical dataset: CervCell-Seg

To further evaluate TiLCA-Net in real-world cervical pathological scenarios, we constructed a private clinical dataset, CervCell-Seg, from cervical pathology cases collected at Shenzhen People’s Hospital. The dataset contained 47 H&E-stained image tiles from 20 independent female patients. According to the clinical records, patient age ranged from 34 to 78 years, with a mean age of 52.0±13.0 years and a median age of 50 years. The specimens were collected between May 10, 2022 and December 21, 2023. All private clinical data were de-identified before analysis. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This study was approved by the Clinical Research Ethics Committee of Shenzhen People’s Hospital (approval No. LL-KY-2025300-01), and informed consent was waived because of the retrospective and de-identified nature of the study.

The diagnostic spectrum was dominated by squamous carcinoma and squamous epithelial lesion-related pathology, with one case of cervical adenocarcinoma. This composition reflects the dominant pathological patterns encountered in cervical cancer practice. The selected cases covered heterogeneous tumor morphology, inflammatory infiltration, stromal components, and variable nuclear appearances, providing clinically relevant morphological diversity for model evaluation. All selected slides were prepared using routine H&E-staining according to the standard clinical pathology workflow. WSIs were digitized using an SQS-600P digital pathology scanning system (TEKSQRAY, Shenzhen Shengqiang Technology Co., Ltd., Shenzhen, China) and viewed with DPVIEW software (version V2.0.11.20240801). The slides were scanned at 40× magnification, corresponding to a spatial resolution of 0.205915 μm/pixel. The scanner metadata recorded an objective lens model of UPlanXAPO20X and an instrument identifier of SQS600P-20220004. From these WSIs, representative image tiles of 1,000×1,000 pixels were selected for annotation and analysis.

Nuclear instance boundaries and category labels were annotated under the supervision of expert pathologists and stored in JSON format. To improve annotation reliability, all annotations were independently reviewed by two experienced pathologists before final inclusion. Cases with inconsistent interpretations were re-evaluated and resolved through consensus discussion. By integrating pathological diagnostic criteria with computer-vision requirements, the annotated nuclei were categorized into four classes: tumor, normal, inflammatory, and spindle-shaped. The final dataset contained 12,279 annotated nuclear instances, including 1,690 tumor nuclei (13.76%), 6,094 normal nuclei (49.63%), 3,629 inflammatory nuclei (29.56%), and 866 Spindle-shaped nuclei (7.05%), as shown in Figure 4B. The main clinicopathological characteristics, imaging parameters, data split, and class distribution of CervCell-Seg are summarized in Table 1. Representative annotation examples are shown in Figure 5.

Table 1. Summary of the CervCell-Seg dataset.

Item	Description
Patients	20 female patients
Age, years	Mean ± SD: 52.0±13.0, range, 34–78
Collection period	May 2022–Dec 2023
Image tiles	47 H&E-stained tiles
Split	33 training tiles from 14 patients; 14 testing tiles from 6 patients
Scanner	SQS-600P digital pathology scanning system, 40×, 0.205915 μm/pixel
Tile size	1,000×1,000 pixels
Annotated nuclei	12,279
Nuclear classes	Tumor, normal, inflammatory, spindle-shaped
Class distribution	Tumor: 1,690 (13.76%); normal: 6,094 (49.63%); inflammatory: 3,629 (29.56%); spindle-shaped: 866 (7.05%)

H&E, hematoxylin and eosin; SD, standard deviation.

Figure 5.

Detailed annotation information of 23-28027_1 and 23-41293_3. All images are H&E-stained and acquired at 40× magnification (0.205915 μm/pixel). H&E, hematoxylin and eosin.

Dataset partition and quality control

For CoNSeP, we followed the official dataset partition, using 27 image tiles for training and 14 image tiles for testing. For MoNuSeg, the experimental collection was divided at the source-image level, with 37 annotated images used for training and 14 images used for evaluation. Patch extraction was performed after this division, and no source image appeared in both subsets. For CervCell-Seg, dataset partitioning was performed strictly at the patient level to avoid potential data leakage and overestimation of model performance. Image tiles from the same patient were assigned exclusively to either the training or testing subset. Specifically, 33 image tiles from 14 patients were used for training, while 14 image tiles from the remaining 6 patients were used for testing. No patient overlap existed between the two subsets. The same dataset-specific partitions were used for all comparative experiments to ensure fair model evaluation.

Before annotation and patch extraction, all candidate image regions were reviewed to ensure sufficient tissue content and recognizable nuclear morphology. Image tiles with severe out-of-focus blur, tissue folding, extensive background regions, or severe staining artifacts were excluded. The retained tiles were selected to cover representative cervical pathological structures, including tumor regions, inflammatory microenvironments, stromal components, and heterogeneous nuclear morphologies.

For patch extraction, CervCell-Seg, CoNSeP, and MoNuSeg followed the same sliding-window preprocessing strategy. Specifically, patches were extracted using a window size of 540×540 pixels and a stride of 164×164 pixels. Mirror padding was applied to edge regions to maintain a consistent patch size. This preprocessing protocol was kept identical for TiLCA-Net and all compared baseline models, including HoVer-Net, to ensure a fair comparison.

Implementation details

The proposed network was implemented using the PyTorch framework. The patch-level runtime benchmark reported in this study was measured on a single NVIDIA GeForce RTX 5070 Ti graphics processing unit (GPU) under a unified evaluation script. During training, data augmentation was applied, including random rotations, flipping, and cropping. The training procedure consisted of two stages: in the first stage (50 epochs), only the decoder parameters were updated; in the second stage (150 epochs), all layers were fine-tuned. For the reproduced HoVer-Net (14), SMILE (38), and TiLCA-Net experiments, the same dataset partitions and core training settings were used whenever applicable to ensure fair comparison. Specifically, the Adam optimizer was utilized with an initial learning rate of 10⁻⁴, which decayed by a factor of 0.1 every 25 epochs. To ensure reproducibility, all experiments employed fixed random seeds across PyTorch, NumPy, and CUDA backends, with deterministic convolution algorithms enabled.

Statistical analysis

To comprehensively evaluate the model’s performance, we used the Dice coefficient to quantify pixel-level overlap. However, given the Dice coefficient’s inability to distinguish between individual instances, we used the AJI (46) as a core instance-level metric. The calculation formulas for the Dice coefficient and AJI are presented in Eqs. [16,17], respectively:

$$\begin{array}{c}Dic{e}_{np}=\frac{2{\sum }_i|Y_i\cap Y_i^{,}|}{{\sum }_i(\left|Y_i\right|+\left|Y_i^{,}\right|)}\end{array}$$ [16]

$$\begin{array}{c}AJI=\frac{{\sum }_{i=1}^N|G_i\cap P_{M(i)}|}{{\sum }_{i=1}^N|G_i\cap P_{M(i)}|+{\sum }_{k\in U}\left|P_k\right|}\end{array}$$ [17]

Where Y and Y' denote the foreground pixel sets of the ground truth and predicted masks, respectively. $G_i$ represents the pixel set of the i-th ground truth nucleus, $P_{M(i)}$ denotes its corresponding matched predicted instance, and U represents the set of indices for unmatched (mis-detected) predicted instances.

Regarding panoptic quality (PQ) metrics, adhering to the definition proposed by Kirillov et al. (47), we introduce PQ alongside its two constituent components: detection quality (DQ) and segmentation quality (SQ). The definitions of PQ and its components are presented in Eq. [18]:

$$\begin{array}{c}PQ=\frac{\left|TP\right|}{\left|TP\right|+0.5\left|FP\right|+0.5\left|FN\right|}\times \frac{{\sum }_{(p,g)\in TP}IoU(p,g)}{\left|TP\right|}\end{array}$$ [18]

Where TP (true positives), FP (false positives), and FN (false negatives) denote the counts of successfully matched pairs, false detections, and missed detections, respectively.

For multi-class cellular analysis, we report the overall classification accuracy (Acc, Eq. [19]) and the weighted F1-score (Eq. [20]). Furthermore, given the class imbalance present in the dataset, we compute per-class Dice scores (denoted as Dice_0 through Dice_4, corresponding to the background and specific cell types, respectively, as per Eq. [21]) to independently assess the model’s performance across different categories.

$$\begin{array}{c}Acc=\frac{{\sum }_{i=0}^{C}T{P}_i}{N_{total}}\end{array}$$ [19]

$$\begin{array}{c}{F}_1=\frac{2\ast TP}{2\ast TP+FP+FN}\end{array}$$ [20]

$$\begin{array}{c}Dic{e}_x=\frac{2|P_x\cap G_x|}{\left|P_x\right|+\left|G_x\right|}\end{array}$$ [21]

Where x denotes a specific category (where x∈{0, 1, 2, 3, 4}), while $P_x$ and $G_x$ represent the predicted region and the ground truth region for class x, respectively.

To provide supplementary statistical evidence, we conducted patch-level paired comparisons using 686 matched test patches for each dataset. The patches were paired according to their patch identifiers, and the metric values for each model were first averaged across the available random seeds. Two-sided paired sign-flip permutation tests were performed for Dice, AJI, and PQ. Mean paired differences and 95% bootstrap confidence intervals were calculated, and the p-values were adjusted across the 18 comparisons using the Holm procedure. Because multiple overlapping patches may originate from the same source image, these results are interpreted as exploratory patch-level evidence rather than image-level or patient-level statistical inference.

Results

Experimental results and comparison

To comprehensively validate the performance of TiLCA-Net in nuclei segmentation and panoptic analysis tasks, we conducted comparative experiments against a selection of representative baseline models. The comparative methods encompass classic semantic segmentation architectures, such as U-Net (12), Mask-RCNN (23), DIST (48), and Micro-Net (49), as well as SOTA models in the field of pathological image analysis, including NC-Net (50), CDNet (51), Diffmix (52), and HoVer-Net (14). The experiments were conducted on the public CoNSeP and MoNuSeg datasets and the proprietary clinical CervCell-Seg dataset constructed in this study. For methods reproduced in this study, including HoVer-Net, SMILE, and TiLCA-Net, we used the same dataset partitions and preprocessing protocols described in Section 2.6.3. UN-SAM and HoVer-NeXt (53) were initialized from their official pretrained weights and then fine-tuned on the corresponding training sets. Results for previously published methods that were not reproduced in our environment were taken from the original publications and are reported as single values. Therefore, only the reproduced or newly evaluated methods are reported with mean ± standard deviation.

Table 2 presents the quantitative instance segmentation results on CoNSeP, while Table 3 presents the results on CervCell-Seg and MoNuSeg. Comprehensive evaluation was performed using Dice, AJI, DQ, SQ, and PQ, and the best performance for each metric is marked with an asterisk (*). On CoNSeP, TiLCA-Net achieved Dice, AJI, DQ, SQ, and PQ values of 0.8389±0.0023, 0.5462±0.0029, 0.6885±0.0058, 0.7767±0.0068, and 0.5278±0.0020, respectively. Compared with HoVer-Net, TiLCA-Net improved Dice by 0.0037, AJI by 0.0089, DQ by 0.0410, SQ by 0.0140, and PQ by 0.0236. In particular, the improvement in PQ indicates that TiLCA-Net achieved a better balance between nuclear DQ and SQ. These results suggest that TiLCA-Net provides a more favorable balance between instance detection, boundary delineation, and computational efficiency among the reproduced methods.

Table 2. Comparative results of nuclei segmentation on the CoNSeP dataset.

Method	CoNSeP
	Dice	AJI	DQ	SQ	PQ
QuPath (54)	0.5880	0.2490	0.2160	0.6410	0.1510
U-Net	0.7240	0.4820	0.4880	0.6720	0.3820
Mask-RCNN	0.7400	0.4740	0.6190	0.7400	0.4600
DIST	0.8040	0.5020	0.5440	0.7280	0.3980
Micro-Net	0.7940	0.5270	0.6000	0.7450	0.4490
NC-Net	0.8290	0.5180	0.6310	0.7600	0.4820
CDNet	0.8350	0.5410	0.6520	0.7710	0.5050
Diffmix	0.8360	0.5630*	0.6580	0.7660	0.5050
UN-SAM	0.8371±0.0009	0.2524±0.0003	0.2959±0.0024	0.7497±0.0024	0.2232±0.0019
HoVer-NeXt	0.7828±0.0002	0.3344±0.0007	0.2611±0.0014	0.7253±0.0010	0.1929±0.0010
HoVer-Net	0.8352±0.0023	0.5373±0.0015	0.6475±0.0091	0.7627±0.0016	0.5042±0.0109
SMILE	0.8285±0.0098	0.5039±0.0047	0.6703±0.0042	0.7752±0.0005	0.5263±0.0035
TiLCA-Net	0.8389±0.0023*	0.5462±0.0029	0.6885±0.0058*	0.7767±0.0068*	0.5278±0.0020*

Data are presented as value or mean ± standard deviation. *, the best performance for each metric. AJI, Aggregated Jaccard Index; DQ, detection quality; PQ, panoptic quality; SQ, segmentation quality.

Table 3. Comparative results of nuclei segmentation on the CervCell-Seg and MoNuSeg datasets.

Dataset	Method	Dice	AJI	DQ	SQ	PQ
CervCell-Seg	UN-SAM	0.8524±0.0030	0.4599±0.0056	0.4957±0.0055	0.7656±0.0087	0.3837±0.0084
	HoVer-NeXt	0.8148±0.0005	0.4360±0.0084	0.3945±0.0101	0.7367±0.0081	0.2946±0.0104
	HoVer-Net	0.8481±0.0032	0.5446±0.0052	0.7593±0.0042	0.7559±0.0046	0.5780±0.0034
	SMILE	0.8424±0.0033	0.5509±0.0048	0.7616±0.0021	0.7756±0.0006	0.5907±0.0013
	TiLCA-Net	0.8541±0.0006*	0.5534±0.0027*	0.7635±0.0018*	0.7791±0.0022*	0.5915±0.0033*
MoNuSeg	UN-SAM	0.8332±0.0029*	0.4840±0.0346	0.6763±0.0280	0.7551±0.0030	0.5113±0.0229
	HoVer-NeXt	0.8133±0.0013	0.6477±0.0019*	0.8147±0.0023*	0.7456±0.0024	0.6079±0.0003*
	HoVer-Net	0.8040±0.0031	0.5480±0.0093	0.7318±0.0073	0.8080±0.0009	0.5627±0.0059
	SMILE	0.8035±0.0090	0.5497±0.0213	0.7851±0.0250	0.7970±0.0009	0.5479±0.0183
	TiLCA-Net	0.8153±0.0016	0.5641±0.0046	0.7915±0.0059	0.8093±0.0102*	0.5779±0.0033

Data are presented as mean ± standard deviation. *, the best performance for each metric. AJI, Aggregated Jaccard Index; DQ, detection quality; PQ, panoptic quality; SQ, segmentation quality.

On the CervCell-Seg dataset (Table 3), TiLCA-Net further evaluates its performance in a proprietary clinical scenario. TiLCA-Net achieved Dice, AJI, DQ, SQ, and PQ values of 0.8541±0.0006, 0.5534±0.0027, 0.7635±0.0018, 0.7791±0.0022, and 0.5915±0.0033, respectively. Compared with HoVer-Net, TiLCA-Net improved all five metrics, with absolute gains of 0.0060 in Dice, 0.0088 in AJI, 0.0042 in DQ, 0.0232 in SQ, and 0.0135 in PQ. Compared with SMILE, TiLCA-Net also achieved higher Dice, AJI, DQ, SQ, and PQ, although the improvement in PQ was relatively small. These results indicate that TiLCA-Net maintains competitive segmentation performance in the available CervCell-Seg cohort. On MoNuSeg, TiLCA-Net achieved a Dice score of 0.8153, an AJI of 0.5641, a DQ of 0.7915, an SQ of 0.8093, and a PQ of 0.5779. It outperformed HoVer-Net and SMILE across all five-segmentation metrics. The recent pretrained methods exhibited complementary strengths: UN-SAM achieved a higher Dice score, whereas HoVer-NeXt obtained higher AJI, DQ, and PQ values. TiLCA-Net achieved the highest SQ among the compared methods, indicating accurate delineation of matched nuclear instances.

Table 4 details the quantitative performance of TiLCA-Net in the multi-class nuclei classification task. Overall, the model achieved consistent improvements in both classification accuracy (NC-ACC) and weighted F1-score (NC-F1) across both datasets. Specifically, on CoNSeP, NC-ACC increased by 1.9 percentage points to 0.9128, with NC-F1 reaching 0.6561. On CervCell-Seg, NC-ACC and NC-F1 reached 0.9460 and 0.7434, respectively; these results indicate that the proposed TSEncoder played a positive role in mitigating multi-task feature conflicts and ameliorating negative transfer. Furthermore, at a fine-grained level, the model demonstrated a balanced classification capability: it outperformed HoVer-Net in specific categories on the CoNSeP dataset [such as Dice_0 (0.9551±0.0016) and Dice_2 (0.7313±0.0147)] and similarly surpassed the comparative baseline on the CervCell-Seg dataset, achieving 0.6115±0.0250 for Dice_1 and 0.4741±0.0114 for Dice_4 on CervCell-Seg. On the CoNSeP dataset, TiLCA-Net exhibits marginal decreases in Dice_3 and Dice_4, corresponding to the Inflammatory and Miscellaneous categories. These classes are characterized by high morphological heterogeneity and ambiguous boundaries, where the task-specialized attention strategy may allocate relatively less capacity compared to a shared encoder that implicitly averages features across all classes. Nevertheless, the overall classification accuracy and F1 score remain superior, indicating that the performance trade-off is localized to minority categories and does not compromise global discriminative ability.

Table 4. Experimental results of nuclei classification on the CoNSeP and CervCell-Seg datasets.

Dataset	Models	ACC		F1	Dice
		NS	NC	NC	Dice_0	Dice_1	Dice_2	Dice_3	Dice_4
CoNSeP	HoVer-Net	0.9251±0.0043	0.8939±0.0112	0.6535±0.0014	0.9506±0.0012	0.5305±0.0092	0.7094±0.0148	0.8292±0.0023*	0.6860±0.0056*
	SMILE	0.9326±0.0008	0.9091±0.0010	0.6493±0.0078	0.9528±0.0011	0.5359±0.0039	0.6870±0.0025	0.8036±0.0078	0.6595±0.0131
	TiLCA-Net	0.9371±0.0005*	0.9128±0.0023*	0.6561±0.0119*	0.9551±0.0016*	0.5417±0.0040*	0.7313±0.0147*	0.8163±0.0081	0.6733±0.0178
CervCell-Seg	HoVer-Net	0.9634±0.0002	0.9351±0.0006	0.7263±0.0004	0.9786±0.0003	0.5506±0.0195	0.6761±0.0025	0.6062±0.0008	0.4661±0.0060
	SMILE	0.9656±0.0010	0.9390±0.0035	0.7359±0.0012	0.9791±0.0008	0.5663±0.0265	0.6539±0.0249	0.6211±0.0046	0.4619±0.0031
	TiLCA-Net	0.9673±0.0002*	0.9460±0.0009*	0.7434±0.0023*	0.9802±0.0005*	0.6115±0.0250*	0.6858±0.0460*	0.6460±0.0178*	0.4741±0.0114*

Data are presented as mean ± standard deviation. *, the best performance for each metric. ACC, accuracy; NC, nuclear classification; NS, nuclear segmentation.

Overall, TiLCA-Net demonstrates robust performance in multi-classification tasks while maintaining high instance segmentation precision, thereby providing a viable auxiliary reference for clinical pathological diagnosis. Figures 6,7 illustrate the qualitative analysis results of the TiLCA-Net model on the CoNSeP and CervCell-Seg datasets, respectively.

Figure 6.

Qualitative results of the TiLCA-Net model on the CoNSeP dataset. (A) Input images; (B) prediction results of TiLCA-Net; (C) prediction results of HoVer-Net. All images are H&E-stained and acquired at 40× magnification (0.205915 μm/pixel). H&E, hematoxylin and eosin.

Figure 7.

Qualitative results of the TiLCA-Net model on the CervCell-Seg dataset. (A) Input images; (B) Ground truth annotations; (C) Prediction results of TiLCA-Net. All images are H&E-stained and acquired at 40× magnification (0.205915 μm/pixel). H&E, hematoxylin and eosin.

As shown in Table 5, the per-class classification results on CervCell-Seg further reveal several limitations of TiLCA-Net. The rare Spindle-shaped class achieved a sensitivity of 0.6289±0.0879 and an F1-score of 0.5252±0.1081, which were lower and more variable than those of the Normal and Inflammatory classes. Its high specificity of 0.9808±0.0072 indicates that the model rarely assigned other nuclei to this category, but it also suggests conservative prediction of spindle-shaped nuclei. This difficulty can be attributed to both the limited number of spindle-shaped samples and their morphological similarity to other elongated stromal nuclei. The Tumor class achieved high sensitivity (0.9500±0.0283) but a more moderate F1-score (0.6502±0.1057), indicating that some morphologically similar nuclei were incorrectly classified as Tumor. Qualitative inspection of erroneous predictions further indicated that densely clustered nuclei, indistinct inter-nuclear boundaries, weak staining, and tissue artifacts were common sources of instance merging, missed nuclei, and false-positive predictions.

Table 5. CervCell-Seg per-class classification performance.

Nuclear class	Sensitivity	Specificity	F1-score	Balanced accuracy
Tumor (type 1)	0.9500±0.0283	0.9395±0.0176	0.6502±0.1057	0.9448±0.0100
Normal (type 2)	0.7793±0.0452	0.8013±0.0509	0.8116±0.0238	0.7903±0.0102
Inflammatory (type 3)	0.7522±0.0745	0.9109±0.0311	0.7786±0.0449	0.8315±0.0229
Spindle-shaped (type 4)	0.6289±0.0879	0.9808±0.0072	0.5252±0.1081	0.8048±0.0443

Data are presented as mean ± standard deviation. Balanced accuracy was defined as the average of sensitivity and specificity. The matched-only protocol evaluates classification after instance matching and does not include missed or spurious nuclei in these classification metrics.

Ablation studies

To systematically validate the effectiveness and individual contributions of the TSEncoder, the ICE module, and the LCA-Decoder within TiLCA-Net, we conducted a detailed ablation study on the CervCell-Seg dataset. The experiment was designed with a lightweight shared-encoder network as the baseline (M1), followed by a stronger hard-parameter-sharing baseline (M2) and progressive integration of the proposed TSEncoder, ICE module, and LCA-Decoder. We evaluated the ablation variants in terms of segmentation performance, computational cost, runtime efficiency, and classification performance. The results are summarized in Tables 6-8, and the component-level changes in PQ and floating point operations per second (FLOPs) are visualized in Figure 8. In Table 6, M1 and M2 do not contain the proposed modules; M1 is a lightweight shared-encoder baseline, whereas M2 is a stronger hard-parameter-sharing baseline with higher encoder capacity.

Table 6. Ablation study and computational complexity comparison.

Model	TSEncoder	ICE	LCA-Decoder	FLOPs (G)	Params (M)	Dice	AJI	DQ	SQ	PQ
M1	−	−	−	60.32	30.63	0.8407±0.0020	0.5272±0.0014	0.7423±0.0008	0.7745±0.0015	0.5749±0.0016
M2	−	−	−	98.75	73.20	0.8429±0.0066	0.5519±0.0011	0.7605±0.0012	0.7757±0.0029	0.5880±0.0035
M3	✓	−	−	98.76	73.23	0.8450±0.0016	0.5567±0.0055	0.7627±0.0044	0.7752±0.0008	0.5899±0.0016
M4	✓	✓	−	103.53	78.73	0.8458±0.0101	0.5551±0.0065	0.7641±0.0042	0.7765±0.0019	0.5922±0.0047
M5	✓	✓	✓	75.48	83.25	0.8534±0.0007	0.5455±0.0024	0.7652±0.0032	0.7787±0.0005	0.5924±0.0028

Data are presented as value or mean ± standard deviation. ✓, component included; −, component excluded. AJI, Aggregated Jaccard Index; DQ, detection quality; FLOP, floating point operations per second; ICE, iterative cross-enhancement; LCA, lightweight context aggregation; Params, parameters; PQ, panoptic quality; SQ, segmentation quality.

Table 7. Patch-level runtime comparison on an NVIDIA GeForce RTX 5070 Ti.

Model	Latency BS1 (ms/patch)	Throughput BS8 (patches/s)	Peak allocated memory BS8 (MiB)
HoVer-Net	17.420	51.274	2,649.370
TiLCA-Net	20.531	60.136	3,069.640

BS1, batch size 1; BS8, batch size 8; MiB, mebibyte.

Table 8. Classification performance of component configurations on CervCell-Seg.

Model	Acc_NS	Acc_NC	F1_NC	Dice_0	Dice_1	Dice_2	Dice_3	Dice_4
M1	0.9650±0.0003	0.9413±0.0004	0.7423±0.0008	0.9799±0.0000	0.5570±0.0122	0.6723±0.0042	0.6214±0.0281	0.4286±0.0319
M2	0.9655±0.0009	0.9456±0.0090	0.7505±0.0012	0.9793±0.0007	0.5255±0.1673	0.6790±0.0596	0.6532±0.0420	0.4798±0.0313
M3	0.9657±0.0004	0.9479±0.0023	0.7541±0.0042	0.9800±0.0005	0.6500±0.0758	0.6792±0.0212	0.6128±0.0150	0.4753±0.0769
M4	0.9666±0.0017	0.9489±0.0071	0.7527±0.0044	0.9794±0.0012	0.6375±0.0900	0.6791±0.0354	0.6644±0.0739	0.4700±0.0857
M5	0.9674±0.0007	0.9487±0.0049	0.7616±0.0137	0.9884±0.0003	0.6179±0.1805	0.6852±0.0079	0.6667±0.0560	0.4816±0.0941

Data are presented as mean ± standard deviation. Acc, accuracy; NC, nuclear classification; NS, nuclear segmentation.

Figure 8.

Quantitative ablation analysis of component effectiveness. The blue bars indicate PQ, and the red dashed line represents FLOPs. FLOP, floating point operations per second; ICE, iterative cross-enhancement; PQ, panoptic quality.

Computational complexity and runtime analysis (Tables 6,7): Table 6 compares FLOPs and parameter counts across different ablation configurations, while Table 7 reports patch-level runtime measurements on an NVIDIA GeForce RTX 5070 Ti. The complete model achieved 75.48 G FLOPs and 83.25 M parameters. Compared with HoVer-Net, TiLCA-Net required fewer FLOPs and achieved higher batch throughput under the unified benchmark setting. However, the runtime measurements also showed higher single-patch latency and peak GPU memory consumption. These results indicate that TiLCA-Net improves batch-processing efficiency, but the practical efficiency gain should be interpreted as a trade-off rather than a uniform acceleration across all runtime indicators.

Table 6 demonstrates the effects of progressively introducing the proposed components. M1 served as a lightweight shared-encoder baseline, whereas M2 represented a stronger hard-parameter-sharing baseline with higher encoder capacity. Compared with M2, introducing the task-specialized encoder branches in M3 improved Dice from 0.8429±0.0066 to 0.8450±0.0016, AJI from 0.5519±0.0011 to 0.5567±0.0055, DQ from 0.7605±0.0012 to 0.7627±0.0044, and PQ from 0.5880±0.0035 to 0.5899±0.0016. These improvements suggest that task-specialized feature extraction helps reduce feature conflict among segmentation, instance separation, and classification tasks. Further adding the ICE module in M4 increased DQ to 0.7641±0.0042 and PQ to 0.5922±0.0047, indicating that cross-task feature enhancement contributes to more reliable nuclear detection and instance-level quality. Finally, the complete model M5 incorporated the LCA-Decoder and achieved the highest Dice (0.8534±0.0007), SQ (0.7787±0.0005), and PQ (0.5924±0.0028), while reducing FLOPs from 103.53 G in M4 to 75.48 G. These results indicate that the LCA-Decoder improves the accuracy-complexity trade-off by reducing theoretical computation while maintaining competitive instance segmentation performance.

Table 8 evaluates the classification performance of different component configurations on CervCell-Seg. Compared with the hard-parameter-sharing baseline M2, introducing the task-specialized encoder in M3 increased Acc_NC from 0.9456±0.0090 to 0.9479±0.0023 and F1_NC from 0.7505±0.0012 to 0.7541±0.0042. Further incorporating the ICE module in M4 maintained competitive overall classification performance and improved Dice_3 from 0.6128±0.0150 to 0.6644±0.0739, suggesting improved recognition of inflammatory nuclei under complex tissue backgrounds. The complete model M5 achieved the highest Acc_NS (0.9674±0.0007) and F1_NC (0.7616±0.0137), while obtaining Dice_1 of 0.6179±0.1805 and Dice_4 of 0.4816±0.0941. These results indicate that the proposed components improve overall classification performance, although rare categories such as spindle-shaped nuclei still show relatively large variability and remain challenging.

Patch-level statistical comparison

Figure 9 summarizes the patch-level paired comparisons of TiLCA-Net with HoVer-Net and SMILE. On CoNSeP, TiLCA-Net achieved significant improvements over HoVer-Net in Dice, AJI, and PQ after Holm correction. On MoNuSeg, the improvements in Dice and PQ were significant, whereas the AJI difference was not significant. On CervCell-Seg, TiLCA-Net produced positive mean differences over HoVer-Net for all three metrics, but these differences did not remain statistically significant after multiple-comparison correction.

Figure 9.

Patch-level paired comparisons of TiLCA-Net with HoVer-Net and SMILE. Points and horizontal bars indicate mean differences and 95% CIs, respectively. Significance was determined using Holm-adjusted paired sign-flip tests: ns, not significant; *, P<0.05; **, P<0.01; ***, P<0.001. Because the tests were based on overlapping image patches, the results should be interpreted as supplementary patch-level evidence rather than independent patient-level evidence. AJI, Aggregated Jaccard Index; CI, confidence interval; PQ, panoptic quality.

Compared with SMILE, TiLCA-Net achieved significant improvements in AJI and PQ on CervCell-Seg, whereas the Dice difference was not significant after Holm correction. Significant improvements in Dice, AJI, and PQ were observed on both CoNSeP and MoNuSeg. These findings provide supplementary evidence regarding the consistency of the observed patch-level differences. Because the paired tests were performed on overlapping image patches, the P values and confidence intervals should be interpreted as supplementary patch-level evidence rather than independent patient-level statistical evidence.

Discussion

The experimental results demonstrate that TiLCA-Net achieves competitive performance with lower theoretical computational cost. TiLCA-Net achieved PQ scores of 0.528, 0.578, and 0.592 on CoNSeP, MoNuSeg, and CervCell-Seg, respectively, outperforming HoVer-Net on all three datasets. These results validate the effectiveness of our proposed multi-scale attention injection strategy and efficient decoding architecture. The observed performance gains can be attributed to three synergistic mechanisms. First, the TSEncoder’s hierarchical attention deployment—SAB for shallow texture features, GGCA for mid-level geometric modeling, and CAB for deep semantic abstraction—aligns with the natural feature evolution in deep networks, effectively mitigating negative transfer (35,38). Second, the ICE module’s closed-loop iteration with median pooling demonstrates effectiveness in clinical WSIs containing staining artifacts and background interference. Third, the LCA-Decoder reduces the FLOPs of the complete TiLCA-Net configuration while maintaining competitive segmentation performance through depthwise separable convolutions and multi-scale feature aggregation.

It is worth noting that TiLCA-Net reduces FLOPs but has a larger parameter count than HoVer-Net. This increase primarily stems from the three task-specialized encoder branches, which are introduced to mitigate feature conflict among segmentation, instance separation, and classification tasks. In patch-level benchmarking on an NVIDIA GeForce RTX 5070 Ti, TiLCA-Net achieved higher batch throughput than HoVer-Net, but it also showed higher single-patch latency and peak GPU memory consumption. Therefore, the efficiency advantage of TiLCA-Net should be interpreted as improved batch-processing throughput under the tested setting, rather than a proportional speed-up implied by FLOPs alone.

While TiLCA-Net demonstrates improvements over HoVer-Net, it is important to contextualize these results. Recent approaches such as CellViT (15) leverage Vision Transformers to capture long-range dependencies; however, transformer-based methods typically require substantially larger computational resources. Direct quantitative comparison on CoNSeP is currently infeasible, as CellViT was evaluated on PanNuke under substantially different training protocols (ViT-SAM backbone pre-trained on 104M histological patches). TiLCA-Net offers a complementary CNN-based solution that emphasizes the balance between segmentation performance and computational complexity, while further deployment-oriented optimization remains necessary before routine use on clinical workstations.

Several limitations should be acknowledged: (I) although the evaluation includes CoNSeP, MoNuSeg, and CervCell-Seg, further validation on larger and more diverse public datasets, such as PanNuke and MoNuSAC, is still required; (II) CervCell-Seg is a relatively small single-institution dataset, which limits broad clinical generalizability; validation on larger multi-center cohorts is therefore needed; (III) patch-level runtime and statistical results should be interpreted cautiously, as whole-slide runtime depends on tiling settings and overlapping patches do not represent independent patient-level evidence. Future work will address these through extended multi-tissue evaluation, multi-center clinical validation, and deployment optimization via quantization and pruning.

TiLCA-Net’s reduced FLOPs and higher batch throughput suggest potential utility for batch processing of WSI-derived patches. Nevertheless, practical deployment will require further optimization and validation under whole-slide inference workflows, including tiling strategy, post-processing time, memory scheduling, and hardware-specific acceleration. Accurate NC provides the foundation for deriving prognostic biomarkers such as tumor-infiltrating lymphocyte density scores, nuclear pleomorphism indices, and spatial heterogeneity metrics for TME characterization.

Conclusions

To address feature alignment bias, limited long-range context modeling, and computational redundancy in H&E-stained pathological image analysis, this paper proposes TiLCA-Net, a multi-task network designed to balance segmentation accuracy and computational cost. Through a systematic architectural design, the proposed method strives to strike a better balance between accuracy and efficiency in nuclei instance segmentation and fine-grained classification tasks.

First, to mitigate the issues of feature conflict and negative transfer inherent in multi-task learning, we designed the TSEncoder mechanism. By embedding spatial, dynamic global, and channel attention at the feature extraction level, this module facilitates the differentiated representation of branch features, thereby improving the alignment between feature flows and specific task requirements. Second, targeting noise interference within complex TMEs, we introduced the ICE module. Leveraging a closed-loop iteration of CCA and a median robust enhancement mechanism, this module bolsters the model’s capacity to model global geometric consistency. Finally, we constructed an LCA-Decoder, which employs a hierarchical progressive feature aggregation strategy to minimize computational redundancy while preserving high-resolution details for fine-grained reconstruction.

Experimental evaluations on the public CoNSeP and MoNuSeg datasets and the proprietary clinical CervCell-Seg dataset indicate that TiLCA-Net achieves competitive performance across multiple evaluation metrics, demonstrating advantages over baseline models such as HoVer-Net. Notably, in cervical cancer nuclei classification tests derived from real-world clinical scenarios, the model demonstrates improved recognition capability for challenging samples—such as inflammatory cells with high morphological similarity—benefiting from the microenvironmental context modeling provided by the ICE module. Furthermore, ablation studies verified the effectiveness of individual components. While maintaining competitive PQ, TiLCA-Net reduces FLOPs relative to HoVer-Net and achieves higher batch throughput in patch-level benchmarking, with increased single-patch latency and memory usage.

In summary, TiLCA-Net highlights the performance potential of nuclei panoptic analysis under a computationally efficient design. Its performance across public benchmarks and the available private clinical cohort suggests potential utility for auxiliary computational pathology analysis, although larger multi-center validation, whole-slide-level evaluation, and deployment optimization are needed before clinical application.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This study was approved by the Clinical Research Ethics Committee of Shenzhen People’s Hospital (approval No. LL-KY-2025300-01), and informed consent was waived because of the retrospective and de-identified nature of the study.

Data Sharing Statement

Available at https://qims.amegroups.com/article/view/10.21037/qims-2026-0938/dss

qims-16-10-811-dss.pdf ^{(75.7KB, pdf)}

DOI: 10.21037/qims-2026-0938