Algorithm 1: Developed Q-learning Channel Prediction Structure.

Initialize the $\mathit{Q}$ table values and initialize the reward matrix R with zeroes. Inputs 2. Number of Iterations and the size for the channel parameters for every user device. 3. Initial distance $“{\mathit{d}}_i”$ of every user device from the BS. 4. Path loss parameter $“\mathit{\vartheta }”$. 5. Design random pilot symbols. 6.Initialize the random channel parameters for each user $“{\mathit{h}}_{ij}”$ based on fading model, $j\in \left[1, 2, \dots ,N\right]$ and $i\in \left[1, 2, \dots ,M\right]$. $N$ is the number of antennas at BS and $M$ is the number of devices in the cell. 7. Designate the power percentage $“{\mathit{\eta }}_i”$ for each user. 8.Determine system bandwidth $“\mathit{B}”$, Total transmit power $“{\mathit{P}}_T”$, and noise spectral density $“{\mathit{N}}_o”$ 9. Assign the desired channel parameters $“{\mathit{h}}_{id}”$ and the target rate $“{\mathit{R}}_T”$ Procedure 10.Based on the channel gain ${\left|{\mathit{h}}_{ij}\right|}^2$, total transmit power $“{\mathit{P}}_T”$, and initial power factor for each user $“{\mathit{\eta }}_i”$, signal to interference noise ratio $“\mathit{S}\mathit{I}\mathit{N}{\mathit{R}}_i”$, minimum required rate $“{\mathit{R}}_i”$ can be calculated for each device. 11.At each iteration, compare the initial generated rate $“{\mathit{R}}_i”$ with the target rate $“{\mathit{R}}_T”$. 12.Update the values for the Q-table that represent the current state and action pair $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$. Q-algorithm 13.identify discount factor $“\mathit{\gamma }”$, learning rate $“\mathit{\alpha }”$, the current state, and the terminal state. 14. Choose the next state at random and set it as the next new state. 15. Inspect all possible actions $“{\mathit{a}}_i”$ to move to the new state. 16.Select the best action ${\mathit{a}}_i\in \mathit{A}$, which satisfies the maximum value for the Q-value function argmax $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$ to move to the new state. 17Identify the immediate Reward $“\mathit{R}”$, based on the action implemented to move to the new state. 18.Based on the following: (1) maximum Q-value $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$ obtained in (16), (2) the corresponding reward $“\mathit{R}”$, (3) the discount factor $“\mathit{\gamma }”$, then $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$ can be updated based on bellman’s equation $\mathit{Q}\left(\mathit{s},\mathit{a}\right)\leftarrow \mathit{R}+\mathit{\gamma } \mathrm{argmax} \mathit{Q}\left(\mathit{s},\mathit{a}\right)$ Outputs 19.Based on the updated $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$ values in Q-table, the channel coefficients $“{\mathit{h}}_{ij}”$ and channel gain ${\left|{\mathit{h}}_{ij}\right|}^2$ can be updated and a new user rate can be calculated and compared to the target rate $“{\mathit{R}}_T”$. 20.Compute the difference $“\mathbf{\Delta }\mathit{Q}”$ between the updated value function ${\mathit{Q}}_{\mathit{n}\mathit{e}\mathit{w}}\left(\mathit{s},\mathit{a}\right)$ and the previous $\mathit{Q}\left(\mathit{s},\mathit{a}\right)$. 21.Based on $\left(20\right), \mathit{Q}\left(\mathit{s},\mathit{a}\right)$ value in the Q-table can be further updated according to $\mathit{Q}\left(\mathit{s},\mathit{a}\right)\leftarrow \mathit{Q}\left(\mathit{s},\mathit{a}\right)+\alpha ·\mathbf{\Delta }\mathit{Q}$ 22. Check whether the terminal state has been reached or the episode has been completed. 23. Compose predicted channel taps ${\widehat{\mathit{h}}}_i$