Fig. 1.

Overview of the proposed framework. For a given patient CT scan, we sample $k$ instances during training to create a bag as input and feed them through the backbone $F_{\theta }$ to obtain feature maps. Modules $A_{\theta ,S}$ and $A_{\theta ,I}$ learn spatial and instance attention, then perform permutation invariant pooling via $A_{\theta ,I}$ on feature maps (from $A_{\theta ,S}$) to obtain a single patient representative feature. The backbone features are spatially pooled via the 1st $⨂$ operator after $A_{\theta ,S}$, whereas the instance feature aggregation is done with 2nd $⨂$ in $A_{\theta ,I}$, respectively. Prior to pooling (aggregation of all features via $A_{\theta ,I}$), attention-weighted instance features from $A_{\theta ,I}$ are employed for unsupervised contrastive learning as well as patient level learning to obtain the final predictions and update the model.