Table 4:

Equivariant residual connections perform better than aggregated residual connections in both Deep Sets and Set Transformer. Max aggregation for Set Transformer led to exploding gradient so we do not report result.

Path	Residual type	Hematocrit (MSE)	Point Cloud (CE)	Mnist Var (MSE)	Normal Var (MSE)
Deep Sets	equivariant	19.2118 ± 0.0762	0.7096 ± 0.0049	0.3441 ± 0.0036	0.0198 ± 0.0041
	mean	19.3462 ± 0.0260	0.8585 ± 0.0253	1.2808 ± 0.0101	0.8811 ± 0.1824
	max	19.8171 ± 0.0266	0.8758 ± 0.0196	1.3798± 0.0162	0.8964 ± 0.1376
Set Transformer	equivariant	18.6883 ± 0.0238	0.6280 ± 0.0098	0.7921 ± 0.0006	0.0030 ± 0.0000
	mean	19.6945 ± 0.1067	0.8111 ± 0.0453	1.6273 ± 0.0335	0.0147 ± 0.0028