Skip to main content
. 2021 Nov 27;21(23):7925. doi: 10.3390/s21237925
Algorithm 1 Proposed PPO Solution for mmWave HetNet
  • Input: 

     

πθ,Vϕ,{ω1,ω2},Env

  • Instruction: 

     

  • 1:

    for iteration=1,2, ….,do

  • 2:

        for iteration=1,2, …., T do

  • 3:

            for iteration=1,2, …., |LBH|+|N| do

  • 4:

               st=st∪ve

  • 5:

            end for

  • 6:

            at=πθold(st)

  • 7:

            [ret,rdt],st+1=Env(at)

  • 8:

            rt=−ω1·re+ω2·rd

  • 9:

            M=M∪{st,at,rt,st+1}

  • 10:

            A^t= compute advantage estimate from Equation (26)

  • 11:

        end for

  • 12:

        for iteration=1,2, …., K do

  • 13:

            update πθ using Equation (32)

  • 14:

            update Vϕ using Equation (33)

  • 15:

        end for

  • 16:

        θold=θ,ϕold=ϕ

  • 17:

        Drop M

  • 18:

    end for