Figure 9.

Example representation of the AFS network. Note that the E layer has $N/2$ neurons. In the AFS2 configuration, the E layer has $N$ features. As described in the original work, the E layer has a $tanh$ activation function. It serves as input to $N$ attention nets, each of them used to determine the importance value of an input feature. Based on [58].