Algorithm 1 Reinforcement Learning-based Wavelength Allocation Algorithm
Input:   $\mathsf{\Gamma },\epsilon ,{\lambda }_t$
Output:   Wavelength allocation matrix
1:	Initialize Process:
2:	Based on history network data $\mathsf{\Gamma }=\{\{H_1,Q_1\},\{H_2,Q_2\},\cdots ,\{H_T,Q_T\}\}$, the BBU initialize a virtual environment. Then a agent is created, whose action set is $A=\{a_1,a_2,\cdots ,a_L\}$. Initialize $t=1$, $Q_{a_l}=0$ for each $a_l$, the exploration probability $\epsilon$ and and learning rate ${\lambda }_t$.
3:	Learning Process:
4:	for each transmission interval $t=1:T$ do
5:	Generate a random number z
6:	if $z<\epsilon$ then
7:	The agent selects a wavelength allocation strategy $a_t$ with equal probability.
8:	else
9:	The agent selects a wavelength allocation strategy with the maximum $Q-value$
10:	end if
11:	Under $a_t$, $H_t$ and $Q_t$, Algorithm 2 is performed to produce a clustering result
12:	The environment feedbacks the signal $r_{a_t}$ to the agent
13:	The agent makes an update according to Equation (5)
14:	end for