|
Algorithm 1 End-to-End Training Procedure for the FAA Framework |
-
Require:
Training dataset , batch size B, temperature , loss weight , learning rate
-
Ensure:
Trained model parameters
-
Initialise the ResNet-18 backbone with ImageNet-pretrained weights
-
Initialise the FFN parameters randomly
-
Initialise the classifier parameters randomly
-
while the training process has not converged do
-
Sample a mini-batch from
Feature Extraction
-
-
Contrastive Alignment
-
-
Compute the cross-modal similarity matrix:
-
Compute the bidirectional contrastive loss:
Fusion and Classification
-
Fuse the aligned modality representations:
-
Generate class predictions:
-
Compute the classification loss:
Optimisation
-
Compute the total training objective:
-
Update all trainable parameters:
-
end while
-
return trained model parameters
|