Improved Q-Learning Algorithm Based on Approximate State Matching in Agricultural Plant Protection Environment

. 2021 Jun 11;23(6):737. doi: 10.3390/e23060737

Algorithm 3 Selecting a Similar Set Based on Vertical Position

Input: States s
Output: Similar set

S_{v p}

of s
1: Create 2 empty stack:

s t a c k_{1}

s t a c k_{2}

2: while

s \neq \emptyset

do
3: a is the action that pops from the head of state s
4: if

a = f

then
5: Push a to

s t a c k_{1}

6: else if

a = b

then
7: if

s t a c k \neq \emptyset

then
8: Pop an element from

s t a c k_{1}

9: end if
10: else if

a = e

then
11: Clear

s t a c k_{1}

12: else if

a = s_{d}

then
13: Push a to

s t a c k_{2}

and clear

s t a c k_{1}

14: end if
15: end while
16: Measure the length of the

s t a c k_{1}

and

s t a c k_{2}

and mark them as

l_{1}

and

l_{2}

respectively
17:

S_{v p} = \emptyset

18:

S^{'} = S_{p a}

19: while

|S^{'}| \geq θ \cdot |S_{p a}|

do
20: Select

s^{'}

from

S^{'}

21: Calculate

l_{1}^{'}

and

l_{2}^{'}

by executing Line 1 to Line 16 with

s^{'}

22: if

l_{1}^{'} = l_{1}

and

l_{2}^{'} = l_{2}

then
23:

S_{v p} = S_{v p} \cup {s^{'}}

24:

S^{'} = S^{'} ∖ {s^{'}}

25: else
26:

S^{'} = S^{'} ∖ {s^{'}}

27: end if
28: end while
29: Return

S_{v p}