Table 1:

The ablation study of TransMorph models with skip connections and positional embedding. “Conv. skip.” denotes the skip-connections from convolutional layers (indicated by green arrows in Fig. 1); “Trans. skip,” denotes the skip-connections from the Transformer blocks (indicated by orange arrows in Fig. 1); “lrn. positional embedding” denotes the learnable positional embedding; “sin. positional embedding” denotes the sinusoidal positional embedding.

Model	Conv. skip.	Trans. skip.	Parameters (M)
w/o conv. skip.	✓	-	46.70
w/o Trans. skip.	-	✓	41.55
w/o positional embedding	✓	✓	46.77
w/ shuffling	✓	✓	46.77
w/ rel. positional bias	✓	✓	46.77
w/ lrn. positional embedding	✓	✓	63.63
w/ sin. positional embedding	✓	✓	46.77