ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast

Abstract

Medical data often exhibits long-tail distributions with heavy class imbalance, which naturally leads to difficulty in classifying the minority classes (i.e., boundary regions or rare objects). Recent work has significantly improved semi-supervised medical image segmentation in long-tailed scenarios by equipping them with unsupervised contrastive criteria. However, it remains unclear how well they will perform in the labeled portion of data where class distribution is also highly imbalanced. In this work, we present ACTION++, an improved contrastive learning framework with adaptive anatomical contrast for semi-supervised medical segmentation. Specifically, we propose an adaptive supervised contrastive loss, where we first compute the optimal locations of class centers uniformly distributed on the embedding space (i.e., off-line), and then perform online contrastive matching training by encouraging different class features to adaptively match these distinct and uniformly distributed class centers. Moreover, we argue that blindly adopting a constant temperature $\tau$ in the contrastive loss on long-tailed medical data is not optimal, and propose to use a dynamic $\tau$ via a simple cosine schedule to yield better separation between majority and minority classes. Empirically, we evaluate ACTION++ on ACDC and LA benchmarks and show that it achieves state-of-the-art across two semi-supervised settings. Theoretically, we analyze the performance of adaptive anatomical contrast and confirm its superiority in label efficiency.

1. Introduction

With the recent development of semi-supervised learning (SSL) [3], rapid progress has been made in medical image segmentation, which typically learns rich anatomical representations from few labeled data and the vast amount of unlabeled data. Existing SSL approaches can be generally categorized into adversarial training [32,39,16,38], deep co-training [23,43], mean teacher schemes [27,42,14,13,15,7,41,34], multi-task learning [19,11,22,37,35], and contrastive learning [2,29,40,33,24,36].

Contrastive learning (CL) has become a remarkable approach to enhance semi-supervised medical image segmentation performance without significantly increasing the amount of parameters and annotation costs [2,29,36]. In real-world clinical scenarios, since the classes in medical images follow the Zipfian distribution [44], the medical datasets usually show a long-tailed, even heavy-tailed class distribution, i.e., some minority (tail) classes involving significantly fewer pixel-level training instances than other majority (head) classes, as illustrated in Figure 1. Such imbalanced scenarios are usually very challenging for CL methods to address, leading to noticeable performance drop [18].

Fig. 1.

Examples of two benchmarks (i.e., ACDC and LA) with imbalanced class distribution. From left to right: input image, ground-truth segmentation map, class distribution chart, training data feature distribution for multiple classes.

To address long-tail medical segmentation, our motivations come from the following two perspectives in CL training schemes [2,36]: ➊ Training objective – the main focus of existing approaches is on designing proper unsupervised contrastive loss in learning high-quality representations for long-tail medical segmentation. While extensively explored in the unlabeled portion of long-tail medical data, supervised CL has rarely been studied from empirical and theoretical perspectives, which will be one of the focuses in this work; ➋ Temperature scheduler – the temperature parameter $\tau$, which controls the strength of attraction and repulsion forces in the contrastive loss [5,4], has been shown to play a crucial role in learning useful representations. It is affirmed that a large $\tau$ emphasizes anatomically meaningful group-wise patterns by group-level discrimination, whereas a small $\tau$ ensures a higher degree of pixel-level (instance) discrimination [28,25]. On the other hand, as shown in [25], group-wise discrimination often results in reduced model’s instance discrimination capabilities, where the model will be biased to “easy” features instead of “hard” features. It is thus unfavorable for long-tailed medical segmentation to blindly treat $\tau$ as a constant hyperparameter, and a dynamic temperature parameter for CL is worth investigating.

In this paper, we introduce ACTION++, which further optimizes anatomically group-level and pixel-level representations for better head and tail class separations, on both labeled and unlabeled medical data. Specifically, we devise two strategies to improve overall segmentation quality by focusing on the two aforementioned perspectives: (1) we propose supervised adaptive anatomical contrastive learning (SAACL) for long-tail medical segmentation. To prevent the feature space from being biased toward the dominant head class, we first pre-compute the optimal locations of class centers uniformly distributed on the embedding space (i.e., off-line), and then perform online contrastive matching training by encouraging different class features to adaptively match these distinct and uniformly distributed class centers; (2) we find that blindly adopting the constant temperature $\tau$ in the contrastive loss can negatively impact the segmentation performance. Inspired by an average distance maximization perspective, we leverage a dynamic $\tau$ via a simple cosine schedule, resulting in significant improvements in the learned representations. Both of these enable the model to learn a balanced feature space that has similar separability for both the majority (head) and minority (tail) classes, leading to better generalization in long-tail medical data. We evaluated our ACTION++ on the public ACDC and LA datasets [1,31]. Extensive experimental results show that our ACTION++ outperforms prior methods by a significant margin and sets the new state-of-the-art across two semi-supervised settings. We also theoretically show the superiority of our method in label efficiency (Appendix A). Code is released at here.

2. Method

2.1. Overview

Problem Statement

Given a medical image dataset $(\mathit{X},\mathit{Y})$, our goal is to train a segmentation model $\mathit{F}$ that can provide accurate predictions that assign each pixel to their corresponding $K$-class segmentation labels.

Setup

Figure 2 illustrates an overview of ACTION++. By default, we build this work upon ACTION pipeline [36], the state-of-the-art CL framework for semi-supervised medical image segmentation. The backbone model adopts the student-teacher framework that shares the same architecture, and the parameters of the teacher are the exponential moving average of the student’s parameters. Hereinafter, we adopt their model as our backbone and briefly summarize its major components: (1) global contrastive distillation pre-training; (2) local contrastive distillation pre-training; and (3) anatomical contrast fine-tuning.

Fig. 2.

Overview of ACTION++: (1) global and local pre-training with proposed anatomical-aware temperature scheduler, (2) our proposed adaptive anatomical contrast fine-tuning, which first pre-computes the optimal locations of class centers uniformly distributed on the embedding space (i.e., off-line), and then performs online contrastive matching training by encouraging different class features to adaptively match these distinct and uniformly distributed class centers with respect to anatomical features.

Global and Local Pre-training

[36] first creates two types of anatomical views as follows: (1) augmented views - ${\mathbf{x}}^1$ and ${\mathbf{x}}^2$ are augmented from the unlabeled input scan with two separate data augmentation operators; (2) mined views - $n$ samples (i.e., ${\mathbf{x}}^3$) are randomly sampled from the unlabeled portion with additional augmentation. The pairs $\left[{\mathbf{x}}^1,{\mathbf{x}}^2\right]$ are then processed by student-teacher networks $\left[F_s,F_t\right]$ that share the same architecture and weight, and similarly, ${\mathbf{x}}^3$ is encoded by $F_t$. Their global latent features after the encoder $\mathit{E}$ (i.e., $\left[{\mathbf{h}}^1,{\mathbf{h}}^2,{\mathbf{h}}^3\right]$) and local output features after decoder $\mathit{D}$ (i.e., $\left[{\mathbf{f}}^1,{\mathbf{f}}^2,{\mathbf{f}}^3\right]$) are encoded by the two-layer nonlinear projectors, generating global and local embeddings ${\mathbf{v}}_g$ and ${\mathbf{v}}_l.\mathbf{v}$ from $F_s$ are separately encoded by the non-linear predictor, producing $\mathbf{w}$ in both global and local manners¹. Third, the relational similarities between augmented and mined views are processed by SoftMax function as follows: ${\mathbf{u}}_s=\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\left({\mathbf{w}}^1,{\mathbf{v}}^3\right)/{\tau }_s\right)}{{\sum }_{n=1}^N\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\left({\mathbf{w}}^1,{\mathbf{v}}_n^3\right)/{\tau }_s\right)},{\mathbf{u}}_t=\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\left({\mathbf{w}}^2,{\mathbf{v}}^3\right)/{\tau }_t\right)}{{\sum }_{n=1}^N\mathrm{e}\mathrm{x}\mathrm{p}\left(\mathrm{s}\mathrm{i}\mathrm{m}\left({\mathbf{w}}^2,{\mathbf{v}}_n^3\right)/{\tau }_t\right)}$, where ${\tau }_s$ and ${\tau }_t$ are two temperature parameters. Finally, we minimize the unsupervised instance discrimination loss (i.e., Kullback-Leibler divergence $𝒦ℒ$) as:

$${ℒ}_{\text{inst}}=𝒦ℒ\left({\mathbf{u}}_s\parallel {\mathbf{u}}_t\right).$$ (1)

We formally summarize the pretraining objective as the equal combination of the global and local ${ℒ}_{\text{inst}}$, and supervised segmentation loss ${ℒ}_{\text{sup}}$ (i.e., equal combination of Dice loss and cross-entropy loss).

Anatomical Contrast Fine-tuning

The underlying motivation for the fine-tuning stage is that it reduces the vulnerability of the pre-trained model to long-tailed unlabeled data. To mitigate the problem, [36] proposed to fine-tune the model by anatomical contrast. First, the additional representation head $\phi$ is used to provide dense representations with the same size as the input scans. Then, [36] explore pulling queries ${\mathbf{r}}_q\in ℛ$ to be similar to the positive keys ${\mathbf{r}}_k^{+}\in ℛ$, and push apart the negative keys ${\mathbf{r}}_k^{-}\in ℛ$. The AnCo loss is defined as follows:

$${ℒ}_{\text{anco}}=\sum _{c\in 𝒞}\sum _{r_q~{ℛ}_q^c}-\log \frac{\exp \left(r_q\cdot r_k^{c,+}/{\tau }_{an}\right)}{\exp \left(r_q\cdot r_k^{c,+}/{\tau }_{an}\right)+{\sum }_{r_k^{-}~{ℛ}_k^c}\exp \left(r_q\cdot r_k^{-}/{\tau }_{an}\right)},$$ (2)

where $𝒞$ denotes a set of all available classes in the current mini-batch, and ${\tau }_{an}$ is a temperature hyperparameter. For class $c$, we select a query representation set ${ℛ}_q^c$, a negative key representation set ${ℛ}_k^c$ whose labels are not in class $c$, and the positive key ${\mathbf{r}}_k^{c,+}$ which is the $c$-class mean representation. Given $𝒫$ is a set including all pixel coordinates with the same size as $R$, these queries and keys can be defined as: ${ℛ}_q^c={\bigcup }_{[i,j]\in 𝒜}\mathbb{1}\left({\mathbf{y}}_{[i,j]}=c\right){\mathbf{r}}_{[i,j]},{ℛ}_k^c={\bigcup }_{[i,j]\in 𝒜}\mathbb{1}\left({\mathbf{y}}_{[i,j]}\ne c\right){\mathbf{r}}_{[i,j]},{\mathbf{r}}_k^{c,+}=\frac{1}{\left|{ℛ}_q^c\right|}{\sum }_{{\mathbf{r}}_q\in {ℛ}_q^c}{\mathbf{r}}_q$. We formally summarize the fine-tuning objective as the equal combination of unsupervised ${ℒ}_{\text{anco}}$, unsupervised cross-entropy loss ${ℒ}_{\text{unsup}}$, and supervised segmentation loss ${ℒ}_{\text{sup}}$. For more details, we refer the reader to [36].

2.2. Supervised Adaptive Anatomical Contrastive Learning

The general efficacy of anatomical contrast on long-tail unlabeled data has previously been demonstrated by the authors of [36]. However, taking a closer look, we observe that the well-trained $\mathit{F}$ shows a downward trend in performance, which often fails to classify tail classes on labeled data, especially when the data shows long-tailed class distributions. This indicates that such well-trained $\mathit{F}$ is required to improve the segmentation capabilities in long-tailed labeled data. To this end, inspired by [17] tailored for the image classification tasks, we introduce supervised adaptive anatomical contrastive learning (SAACL), a training framework for generating well-separated and uniformly distributed latent feature representations for both the head and tail classes. It consists of three main steps, which we describe in the following.

Anatomical Center Pre-computation

We first pre-compute the anatomical class centers in latent representation space. The optimal class centers are chosen as $K$ positions from the unit sphere ${\mathbb{S}}^{d-1}=\left\{v\in {\mathbb{R}}^d:\parallel v{\parallel }_2=1\right\}$ in the $d$-dimensional space. To encourage good separability and uniformity, we compute the class centers ${\left\{{\mathit{\psi }}_c\right\}}_{c=1}^K$ by minimizing the following uniformity loss ${ℒ}_{\text{unif}}$:

$${ℒ}_{\text{unif}}\left({\left\{{\psi }_c\right\}}_{c=1}^K\right)=\sum _{c=1}^K\text{log}\left(\sum _{c^{\prime }=1}^K\text{exp}\left({\psi }_c\cdot {\psi }_{c^{\prime }}/\tau \right)\right).$$ (3)

In our implementation, we use gradient descent to search for the optimal class centers constrained to the unit sphere ${\mathbb{S}}^{d-1}$, which are denoted by ${\left\{{\psi }_c^{\star }\right\}}_{c=1}^K$. Furthermore, the latent dimension $d$ is a hyper-parameter, which we set such that $d\gg K$ to ensure the solution found by gradient descent indeed maximizes the minimum distance between any two class centers [6]. It is also known that any analytical minimizers of Eqn. 3 form a perfectly regular $K$-vertex inscribed simplex of the sphere ${\mathbb{S}}^{d-1}$ [6]. We emphasize that this first step of pre-computation of class centers is completely off-line as it does not require any training data.

Adaptive Allocation

As the second step, we explore adaptively allocating these centers among classes. This is a combinatorial optimization problem and an exhaustive search of all choices would be computationally prohibited. Therefore, we draw intuition from the empirical mean in the K-means algorithm and adopt an adaptive allocation scheme to iteratively search for the optimal allocation during training. Specifically, consider a batch $ℬ=\left\{{ℬ}_1,\cdots ,{ℬ}_K\right\}$ where ${ℬ}_c$ denotes a set of samples in a batch with class label $c$, for $c=1,\cdots ,K$. Define ${\bar{\varphi }}_c\left(ℬ\right)={\sum }_{i\in {ℬ}_c}{\varphi }_i/{∥{\sum }_{i\in {ℬ}_c}{\varphi }_i∥}_2$ be the empirical mean of class $c$ in current batch, where ${\varphi }_i$ is the feature embedding of sample $i$. We compute assignment $\pi$ by minimizing the distance between pre-computed class centers and the empirical means:

$${\pi }^{\star }=\arg \underset{\pi }{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{c=1}^K{∥{\mathit{\psi }}_{\pi \left(c\right)}^{\star }-{\bar{\varphi }}_c∥}_2.$$ (4)

In implementation, the empirical mean is updated using moving average. That is, for iteration $t$, we first compute the empirical mean ${\bar{\varphi }}_c\left(ℬ\right)$ for batch $ℬ$ as described above, and then update by ${\bar{\varphi }}_c\leftarrow (1-\eta ){\bar{\varphi }}_c+\eta {\bar{\varphi }}_c\left(ℬ\right)$.

Adaptive Anatomical Contrast

Finally, the allocated class centers are well-separated and should maintain the semantic relation between classes. To utilize these optimal class centers, we want to induce the feature representation of samples from each class to cluster around the corresponding pre-computed class center. To this end, we adopt a supervised contrastive loss for the label portion of the data. Specifically, given a batch of pixel-feature-label tuples ${\left\{\left({\omega }_i,{\mathit{\varphi }}_i,y_i\right)\right\}}_{i=1}^n$ where ${\omega }_i$ is the i-th pixel in the batch, ${\mathit{\varphi }}_i$ is the feature of the pixel and $y_i$ is its label, we define supervised adaptive anatomical contrastive loss for pixel $i$ as:

$${ℒ}_{\text{aaco}}=\frac{-1}{n}\sum _{i=1}^n\left(\sum _{{\mathit{\varphi }}_i^{+}}\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathit{\varphi }}_i\cdot {\mathit{\varphi }}_i^{+}/{\tau }_{sa}\right)}{{\sum }_{{\mathit{\varphi }}_j}\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathit{\varphi }}_i\cdot {\mathit{\varphi }}_j/{\tau }_{sa}\right)}+{\lambda }_a\mathrm{l}\mathrm{o}\mathrm{g}\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathit{\varphi }}_i\cdot {\mathit{\nu }}_i/{\tau }_{sa}\right)}{{\sum }_{{\mathit{\varphi }}_j}\mathrm{e}\mathrm{x}\mathrm{p}\left({\mathit{\varphi }}_i\cdot {\mathit{\varphi }}_j/{\tau }_{sa}\right)}\right),$$ (5)

where ${\mathit{\nu }}_i={\mathit{\psi }}_{{\pi }^{\star }\left(y_i\right)}^{\star }$ is the pre-computed center of class $y_i$. The first term in Eqn. 5 is supervised contrastive loss, where the summation over ${\varphi }_i^{+}$ refers to the uniformly sampled positive examples from pixels in batch with label equal to $y_i$. The summation over ${\varphi }_j$ refers to all features in the batch excluding ${\varphi }_i$. The second term is contrastive loss with the positive example being the pre-computed optimal class center.

2.3. Anatomical-aware Temperature Scheduler (ATS)

Training with a varying $\tau$ induces a more isotropic representation space, wherein the model learns both group-wise and instance-specific features [12]. To this end, we are inspired to use an anatomical-aware temperature scheduler in both the supervised and the unsupervised contrastive losses, where the temperature parameter $\tau$ evolves within the range $\left[{\tau }^{-},{\tau }^{+}\right]$ for ${\tau }^{+}>{\tau }^{-}$. Specifically, for iteration $t=1,\cdots ,T$ with $T$ being the total number of iterations, we set ${\tau }_t$ as:

$${\tau }_t={\tau }^{-}+0.5(1+\mathrm{c}\mathrm{o}\mathrm{s}(2\pi t/T\left)\right)\left({\tau }^{+}-{\tau }^{-}\right).$$ (6)

3. Experiments

Experimental Setup

We evaluate ACTION++ on two benchmark datasets: the LA dataset [31] and the ACDC dataset [1]. The LA dataset consists of 100 gadolinium-enhanced MRI scans, with the fixed split [29] using 80 and 20 scans for training and validation. The ACDC dataset consists of 200 cardiac cine MRI scans from 100 patients including three segmentation classes, i.e., left ventricle (LV), myocardium (Myo), and right ventricle (RV), with the fixed split² using 70, 10, and 20 patients’ scans for training, validation, and testing. For all our experiments, we follow the identical setting in [42,19,30,29], and perform evaluations under two label settings (i.e., 5% and 10%) for both datasets.

Implementation Details

We use an SGD optimizer for all experiments with a learning rate of 1e-2, a momentum of 0.9, and a weight decay of 0.0001. Following [42,19,30,29] on both datasets, all inputs were normalized as zero mean and unit variance. The data augmentations are rotation and flip operations. Our work is built on ACTION [36], thus we follow the identical model setting except for temperature parameters because they are of direct interest to us. For the sake of completeness, we refer the reader to [36] for more details. We set ${\lambda }_a,d$ as 0.2, 128, and regarding all $\tau$, we use ${\tau }^{+}=1.0$ and ${\tau }^{-}=0.1$ if not stated otherwise. On ACDC, we use the U-Net model [26] as the backbone with a 2D patch size of 256 × 256 and batch size of 8. For pre-training, the networks are trained for 10K iterations; for fine-tuning, 20K iterations. On LA, we use the V-Net [21] as the backbone. For training, we randomly crop 112 × 112 × 80 patches and the batch size is 2. For pre-training, the networks are trained for 5K iterations. For fine-tuning, the networks are for 15K iterations. For testing, we adopt a sliding window strategy with a fixed stride (18 × 18 × 4). All experiments are conducted in the same environments with fixed random seeds (Hardware: Single NVIDIA GeForce RTX 3090 GPU; Software: PyTorch 1.10.2+cu113, and Python 3.8.11).

Main Results

We compare our ACTION++ with current state-of-the-art SSL methods, including UAMT [42], SASSNet [16], DTC [19], URPC [20], MC-Net [30], SS-Net [29], and ACTION [36], and the supervised counterparts (UNet [26]/VNet [21]) trained with Full/Limited supervisions – using their released code. To evaluate 3D segmentation ability, we use Dice coefficient (DSC) and Average Surface Distance (ASD). Table 2 and Table 1 display the results on the public ACDC and LA datasets for the two labeled settings, respectively. We next discuss our main findings as follows. (1) LA: As shown in Table 1, our method generally presents better performance than the prior SSL methods under all settings. Fig. 4 (Appendix) also shows that our model consistently outperforms all other competitors, especially in the boundary region; (2) ACDC: As Table 2 shows, ACTION++ achieves the best segmentation performance in terms of Dice and ASD, consistently outperforming the previous SSL methods across two labeled settings. In Fig. 3 (Appendix), we can observe that ACTION++ can yield the segmentation boundaries accurately, even for very challenging regions (i.e., RV and Myo). This suggests that ACTION++ is inherently better at long-tailed learning, in addition to being a better segmentation model in general.

Table 2.

Quantitative comparison (DSC[%]/ASD[voxel]) for ACDC under two unlabeled settings (5% or 10%). All experiments are conducted as [42,16,19,20,30,29,36] in the identical setting for fair comparisons. The best results are indicated in bold.

	3 Labeled (5%)				7 Labeled (10%)
Method	Average	RV	Myo	LV	Average	RV	Myo	LV
UNet-F [26]	91.5/0.996	90.5/0.606	88.8/0.941	94.4/1.44	91.5/0.996	90.5/0.606	88.8/0.941	94.4/1.44
UNet-L	51.7/13.1	36.9/30.1	54.9/4.27	63.4/5.11	79.5/2.73	65.9/0.892	82.9/2.70	89.6/4.60
UAMT [42]	48.3/9.14	37.6/18.9	50.1/4.27	57.3/4.17	81.8/4.04	79.9/2.73	80.1/3.32	85.4/6.07
SASSNet [16]	57.8/6.36	47.9/11.7	59.7/4.51	65.8/2.87	84.7/1.83	81.8/0.769	82.9/1.73	89.4/2.99
URPC [20]	58.9/8.14	50.1/12.6	60.8/4.10	65.8/7.71	83.1/1.68	77.0/0.742	82.2/0.505	90.1/3.79
DTC [19]	56.9/7.59	35.1/9.17	62.9/6.01	72.7/7.59	84.3/4.04	83.8/3.72	83.5/4.63	85.6/3.77
MC-Net [30]	62.8/2.59	52.7/5.14	62.6/0.807	73.1/1.81	86.5/1.89	85.1/0.745	84.0/2.12	90.3/2.81
SS-Net [29]	65.8/2.28	57.5/3.91	65.7/2.02	74.2/0.896	86.8/1.40	85.4/1.19	84.3/1.44	90.6/1.57
ACTION [36]	87.5/1.12	85.4/0.915	85.8/0.784	91.2/1.66	89.7/0.736	89.8/0.589	86.7/0.813	92.7/0.804
• ACTION++ (ours)	88.5/0.723	86.9/0.662	86.8/0.689	91.9/0.818	90.4/0.592	90.5/0.448	87.5/0.628	93.1/0.700

Table 1.

Quantitative comparison (DSC[%]/ASD[voxel]) for LA under two unlabeled settings (5% or 10%). All experiments are conducted as [42,16,19,20,30,29,36] in the identical setting for fair comparisons. The best results are indicated in bold. VNet-F (fully-supervided) and VNet-L (semi-supervided) are considered as the upper bound and the lower bound for the performance comparison.

	4 Labeled (5%)		8 Labeled (10%)
Method	DSC[%]↑	ASD[voxel]↓	DSC[%]↑	ASD[voxel]↓
VNet-F [21]	91.5	1.51	91.5	1.51
VNet-L	52.6	9.87	82.7	3.26
UAMT [42]	82.3	3.82	87.8	2.12
SASSNet [16]	81.6	3.58	87.5	2.59
DTC [19]	81.3	2.70	87.5	2.36
URPC [20]	82.5	3.65	86.9	2.28
MC-Net [30]	83.6	2.70	87.6	1.82
SS-Net [29]	86.3	2.31	88.6	1.90
ACTION [36]	86.6	2.24	88.7	2.10
• ACTION++ (ours)	87.8	2.09	89.9	1.74

Ablation Study

We first perform ablation studies on LA with 10% label ratio to evaluate the importance of different components. Table 3 shows the effectiveness of supervised adaptive anatomical contrastive learning (SAACL). Table 4 (Appendix) indicates that using anatomical-aware temperature scheduler (ATS) and SAACL yield better performance in both pre-training and fine-tuning stages. We then theoretically show the superiority of our method in Appendix A.

Table 3.

Ablation studies of Supervised Adaptive Anatomical Contrast (SAACL).

Method	DSC[%]↑	ASD[voxel]↓
KCL [9]	88.4	2.19
CB-KCL [10]	86.9	2.47
SAACL (Ours)	89.9	1.74
SAACL (random assign)	88.0	2.79
SAACL (adaptive allocation)	89.9	1.74

Finally, we conduct experiments to study the effects of cosine boundaries, cosine period, different methods of varying $\tau$, and ${\lambda }_a$ in Table 5, Table 6 (Appendix), respectively. Empirically, we find that using our settings (i.e., ${\tau }^{-}=0.1,{\tau }^{+}=1.0,T/\#\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{s}=1.0$, cosine scheduler, ${\lambda }_a=0.2$) attains optimal performance.

4. Conclusion

In this paper, we proposed ACTION++, an improved contrastive learning framework with adaptive anatomical contrast for semi-supervised medical segmentation. Our work is inspired by two intriguing observations that, besides the unlabeled data, the class imbalance issue exists in the labeled portion of medical data and the effectiveness of temperature schedules for contrastive learning on long-tailed medical data. Extensive experiments and ablations demonstrated that our model consistently achieved superior performance compared to the prior semi-supervised medical image segmentation methods under different label ratios. Our theoretical analysis also revealed the robustness of our method in label efficiency. In future, we will validate CT/MRI datasets with more foreground labels and try t-SNE.

Table 4.

Effect of cosine boundaries in with the largest difference between ${\tau }^{-}$ and ${\tau }^{+}$.

${\tau }^{-}$╲${\tau }^{+}$	0.2	0.3	0.4	0.5	1.0
0.07	84.1	85.0	86.9	87.9	89.7
0.1	84.5	85.9	87.1	88.3	89.9
0.2	84.2	84.4	85.8	87.1	87.6