Table 1. Parameter and operation counts for various networks.

The number of neurons in the input, hidden, and output layers are ${\displaystyle d}$, ${\displaystyle n}$, and ${\displaystyle N}$, respectively. Note these counts do not include parameters of the readout layer, since said layer contributes the same number of parameters for each network ($N(n+1)$ and ${\displaystyle Nn}$ with and without a bias) and are for fixed initial states.

Network	Trainable Parameters	State update operations
2-layer fully connected	${\displaystyle n\left(d+1\right)}$	${\displaystyle \mathcal{O}\left(n{d}^2\right)}$
MPN	${\displaystyle n\left(d+1\right)+2}$	${\displaystyle \mathcal{O}\left(n{d}^2\right)}$
MPNpre	${\displaystyle n\left(d+1\right)+2}$	${\displaystyle \mathcal{O}\left(n{d}^2\right)}$
Vanilla RNN	${\displaystyle n\left(n+d+1\right)}$	${\displaystyle \mathcal{O}\left(n^3\right)+\mathcal{O}\left(n{d}^2\right)}$
GRU	${\displaystyle 3n\left(n+d+1\right)}$	${\displaystyle \mathcal{O}\left(3n^3\right)+\mathcal{O}\left(3n{d}^2\right)}$
LSTM	${\displaystyle 4n\left(n+d+1\right)}$	${\displaystyle \mathcal{O}\left(4n^3\right)+\mathcal{O}\left(4n{d}^2\right)}$