Algorithm 1: ViT-DCNN (Vision Transformer with Deformable Convolution) for Lung and Colon Cancer Classification

1: Input: D = {(Xi, Yi)}, α, T, B, θViT, θDConv, N

2: Initialize: θViT, θDConv

3: for epoch = 1 to T do

4:   for batch = 1 to $\left({\displaystyle \frac{\mathrm{N}}{\mathrm{B}}}\right)$ do

5:         Extract mini-batch:

6:                  $\left({\mathrm{X}}_{\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h},}{\mathrm{Y}}_{\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}\right)\leftarrow {\left\{{\mathrm{X}}_{\mathrm{i},}{\mathrm{Y}}_{\mathrm{i}}\right\}}_{\mathrm{i}=\mathrm{b}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$

7:                      Apply Data Augmentation:

8:                       $\left(X_{aug,}{Y}_{aug}\right)\leftarrow \mathrm{A}\mathrm{u}\mathrm{g}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t}\left(X_{batch,}{Y}_{batch}\right)$

9:            Vision Transformer (ViT) Forward Pass:

10:            Patch Embedding:

11:                     ${\mathrm{P}}_{\mathrm{i}}=\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\left({\mathrm{X}}_{\mathrm{i}}\right)·{\mathrm{W}}_{\mathrm{e}\mathrm{m}\mathrm{b}}+{\mathrm{b}}_{\mathrm{e}\mathrm{m}\mathrm{b}}$

12:            Positional Encoding:

13:                    Zi = Pi + PEi

14:            Multi-Head Self-Attention:

15:                    $\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{Q},\mathrm{K},\mathrm{V}\right)=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{\mathrm{Q}{\mathrm{K}}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}}\right)\mathrm{V}$

16:         Feed-Forward Network:

17:                 FFN(Z) = max(0, ZW1 + b1) W2 + b2

18:          Deformable Convolution Forward Pass:

19:           Deformable Convolution:

20:                     $y_{ij}=\sum _{m=1}^M\sum _{n=1}^N{x}_{i+m+∆m_{ij},j+n+∆n_{ij}·w_{mn}}$

21:              Offset Learning:

22:                      $\left[∆m_{ij},∆n_{ij}\right]=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\left({\mathrm{F}}_{\mathrm{i}\mathrm{j}}\right),{\mathrm{F}}_{ij}\in {\mathbb{R}}^C$

23:              Spatial Attention:

24:                      ${\mathrm{F}}_{\mathrm{r}\mathrm{e}\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}}={\mathrm{F}}_{\mathrm{i}\mathrm{j}}·{\mathrm{A}}_{\mathrm{i}\mathrm{j}}$

25:          Hierarchical Feature Fusion (HFF):

26:              Concatenate Vision Transformer and Deformable CNN Features:

27:                     Fconcat = concat(FViT, FDConv)

28:              Squeeze-and-Excitation (SE) Block:

29:                      $\mathrm{s}=\mathsf{\sigma }\left(\mathrm{M}\mathrm{L}\mathrm{P}\right(\mathrm{z}\left)\right)$

30:              Refined Feature Map:

31:                      ${\mathrm{F}}_{\mathrm{s}\mathrm{e}}={\mathrm{F}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}}·\mathrm{s}$

32:          Prediction and Softmax Activation:

33:              Global Average Pooling:

34:                      $\mathsf{{\rm Z}}=\mathrm{G}\mathrm{A}\mathrm{P}\left({\mathrm{F}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}}\right)={\displaystyle \frac{1}{HW}}\sum _{i=1}^H\sum _{j=1}^W{\mathrm{F}}_{\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}}\left(i,j\right)$

35:              Softmax Layer:

36:                      $P_c=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathrm{W}·\mathrm{z}+\mathrm{b})$

37:              Predicted Class:

38:                      $Y^{\prime }\leftarrow argmax\left(P_{softmax}\right)$

39:          Compute Class:

40:              Cross-Entropy Loss:

41:                      $L_{cross}=-\sum _{i=1}^N\sum _{j=1}^K{y}_{ij}log\left(y_{ij}^{\prime }\right)$

42:              Gradient Computation:

43:                      $\nabla {\theta }_{VIT}{L}_{cross},\nabla {\theta }_{DConv}{L}_{cross}$

44:          Parameter Update (Using AdamW optimizer):

45:              Update the Vision Transformer parameters:

46:                      ${\theta }_{VIT}\leftarrow {\theta }_{VIT}-\alpha ·\nabla {\theta }_{VIT}{L}_{cross}$

47:              Update the Deformable CNN parameters:

48:                     ${\theta }_{DComv}\leftarrow {\theta }_{DConv}-\alpha ·\nabla {\theta }_{DConv}{L}_{cross}$

49:         end for

50:  end for

51:  Output:

52:  Trained ViT-Deformable CNN model with updated parameters θViT and θDConv