Algorithm 1 MILP-MDP adaptive routing algorithm

Input: Network parameters, $E_0$, V, $\alpha$, $\beta$, $\gamma$, packetLength Output: Network lifetime, delay, FND, HND Initialize Q-table $Q(s,a)$ for 81 states Initialize exploration rate $\epsilon =0.3$ Define action set $A=\{0.8,0.9,1.0,1.1\}$ for each round r do    Update node energy and alive-node status    if $r=1$ or $rmodech=0$ or event-triggered then      Perform Distance–Energy Weighted K-means clustering      Observe current state $s\in \{1,\dots ,81\}$      Select action A using $\epsilon$-greedy policy      Solve MILP to obtain routing path subject to flow and subtour constraints      Apply scaling factor to adjust routing    end if    Execute data transmission    Compute reward R according to Equation (24)    Observe next state $s^{\prime }$    Update Q-table using Q-learning rule    Update exploration rate $\epsilon$ end for Record performance metrics (FND, HND, delay, energy)