Algorithm 2 Reinforcement learning and blockchain based (RLBC) routing algorithm.

Input: Environment E; Action Space $\mathcal{A}$; Initial State $x_0$; Reward Discount $\gamma$; Learning Rate $\alpha$; Output: Policy $\pi$;  1: $Q_t$(x,a) = 0, P(x,a) = $\frac{1}{\mid \mathcal{A}\left(x\right)\mid }$;  2: x = $x_0$;  3: for $T=1,2,$ … do  4:   a = ${\pi }^p$(x);  5:   r = reward by routing action a;  6:   $x^{\prime }$ = next state by routing action a;  7:   $a^{\prime }$ = $\pi$($x^{\prime }$);  8:   $Q_t$(x,a) = $Q_t$(x,a) + $\alpha$(r+$\gamma Q_t$($x^{\prime }$,$a^{\prime }$) − $Q_t$(x,a));  9:   $\pi$(x) = arg ${\max }_{a_k}{q}_t$(x,$a_k$)$·Q_t$(x,$a_k$);  10:   x = $x^{\prime }$;  11: end for