Algorithm 3. Channel Selection and Optimal Power Allocation Policy

1: Input: Channel State Identifier, Channel Occupancy Predictor, Belief State and Number of SUs 2: Output: Channel Selected, Updated CSI 3: Set Wmax = 0 4: for i = 1 to N do 5: Set Optimal carrier Φ(i)(t) = 1 and Φ(j)(t) = 0, ∀j ∈ {1, …, N}, j ≠ i, where Φ(t) = {Φ(1)(t), Φ(2)(t), …, Φ(N)(t)} 6: Derive Optimal Power allocation P(i) × (t) and A(i) × (t) 7: Compute Expected Idle time of channel and probability that small cell reserves channel r) 8: Compute reward and set Wmax = max{Wmax, W(i) (Φ(i)(t), P(i) × (t), A(i) × (t))} 9: if Wmax = W(i) (Φ(i)(t), P(i) × (t), A(i) × (t)) then 10: Set Optimal carrier Φ × (t) = Φ(t) 11: end if 12: end for 13: Set update for Channel Selection A × (t) = {A(1) × (t), A(2) × (t), …, A(N) × (t)}