Table 2.

Commonly used activation functions in deep learning

Name (fa)	Expression
Sigmoid function	$f_a\left(x\right)=1/\left(1+\exp \left(-x\right)\right)$
ReLu function	$f_a\left(x\right)=\max \left(0,x\right)$
Tanh function	$f_a\left(x\right)=\left(\exp \left(x\right)-\exp (-x)\right)/\left(\exp \left(x\right)+\exp (-x)\right)$