Figure 1:

An illustration of different attention-based aggregators on multiset of node features. Given two distinct multisets H₁ and H₂ that have the same central node feature h_i and the same distribution of node features, aggregators will map h_i to h_i1 and h_i2 for H₁ and H₂. The Original model will get $h_{i1}^{\prime }=h_{i2}^{\prime }$ and fail to distinguish H₁ and H₂, while our Additive and Scaled models can always distinguish H₁ and H₂ with $h_{i1}^{\prime \prime }\ne h_{i2}^{\prime \prime }$ and $h_{i1}^{\prime \prime \prime }\ne h_{i2}^{\prime \prime \prime }$.