Algorithm 1: RL-Based Routing with Entropy & Gini Reward

Input: Current Node $n_i$ $, \mathrm{Neighbor} \mathrm{Set} N_i$

Output: Next Hop $n_{next}$

1:  $\mathrm{Observe} \mathrm{State} S_t=\left(E_{res},Dis{t}_{\sin k},Q_{len}\right)$ //

2: // Action Selection (Epsilon-Greedy)

3:  $\mathrm{IF} random()<\epsilon$ $\mathrm{THEN} \mathrm{Select} \mathrm{random} n_{next}$ $\mathrm{form} N_i$

4:  $\mathrm{ELSE} n_{next}=argma{x}_{n_j}Q\left(S_t,n_j\right)$

5:  $\mathrm{Forward} \mathrm{Packet} \mathrm{to} n_{next}$ $\mathrm{and} \mathrm{Observe} S_{t+1}$

6: // Reward Calculation (Core Innovation)

7:  $\mathrm{Compute} R_{progress}$$\mathrm{and} R_{energy}$ using Equations (23) and (24)

8:  $\mathrm{Compute} R_{equilibrium}$ based on Entropy Equation (27) & Gini Equation (28)

9:  $R_{total}=w1\cdot R_{progress}+w2\cdot R_{energy}+w3\cdot R_{equilibrium}$

10: // Update Q-Value

11:  $Q(S_t,n_{next})=(1-\alpha )\cdot Q(S_t,n_{next})+\alpha \cdot (R_{total}+\gamma \cdot max(Q(S_{t+1})))$