Algorithm 1: DRL–PPO for Multi-Intersection Traffic Signal Control

Input: Environment ε (SUMO network), policy πθ, value function Vφ

Initialize θ, φ randomly

for episode = 1 to 2000 do

Reset environment ε

for t = 1 to 3600 do

Observe state st ← [queue, wait, phase, time_since_change]

Select action at ← πθ(a|st)

Execute at in SUMO

Observe reward

rt = − α1Wt − α2Qt − α3Et + β1Tt

Store transition (st, at, rt, s{t + 1})

end for

Compute advantage

Ât = rt + γVφ(s{t + 1}) − Vφ(st)

Update policy:

θ ← θ + η∇θLPPO(θ, Ât)

Update value function:

φ ← arg minφ (Vφ(st) − Rt)2

end for

Output: Optimized policy πθ*